In this paper we present discuss the task assignment problem for distributed systems. We also show how this problem is very similar to that of clustering transactions for load balancing purposes and for their efficient execution in a distributed environment. The formalization of these problems in terms of a graph theoretic representation of a distributed program, or of a set of related transactions, is given. The cost function which needs to be minimized by an assignment of tasks to processors or of transactions to clusters is detailed, and we survey related work, as well work on the dynamic load balancing problem. Since the task assignment problem is NP-hard, we present three novel heuristic algorithms that we have tested for solving it and compare them to the well-known greedy heuristic. These novel heuristics use neural networks, genetic algorithms and simulated annealing. Both the resulting performance and the computational cost for these algorithms is evaluated on a large number o...