Three routes to unification

The following classic papers propose three different geometrical ways of unifying gravity and electromagnetism.

Hermann Weyl (1918)

Weyl’s paper was published just a couple of years after Einstein’s theory of General Relativity. The referee was Einstein, who appended his comments. A reply of Weyl follows.
Weyl was unhappy with the assumption, which is usually made in general relativity, that parallel transport preserves the norm of vectors (equivalently, the covariant derivative of the metric vanishes, or the connection is metric, or the holonomy of the connection is in the Lorentz group). He saw no fundamental reason to assume that, and wanted to formulate a theory where the connection is not metric. If the connection is not metric, it has more degrees of freedom and the question arises as to what is their interpretation. Weyl chose a specific form for a nonmetric connection, parametrized by four fields, and proposed that these fields be identified with the four components of the electromagnetic potential. To support this interpretation, he gave a beautiful geometrical interpretation to the "local gauge transformations": a rescaling of the metric by a point-dependent factor results in a change of the electromagnetic potential by the gradient of that factor.
Nowadays, it is easy to see why this cannot work. We know that electromagnetic gauge transformations are "internal" transformations and have nothing to do with spacetime geometry. Einstein pointed this out in his criticism in the following way: if the size of objects changed after being transported around a loop in an electromagnetic field, then the spectral lines of atoms that followed different paths could not be the same, in contrast to experience. Another issue is that electromagnetism is a gauge theory for the compact abelian group U(1) whereas Weyl's theory was a gauge theory for the noncompact abelian group R.
Still, Weyl's ideas have been extraordinarily fruitful: local rescalings of the metric (conformal transformations, nowadays also called Weyl transformations) are a very useful tool in general relativity, and few years after publication of the paper, it was understood that after a reinterpreting Weyl's transformations as changes of phase of the wave function of a charged particle, they do indeed correctly describe electromagnetism. Thus in a sense Weyl’s paper is at the root of all modern gauge theories.
A nonabelian version of Weyl’s theory has been discussed here.

Theodor Kaluza (1921)

This is by far the best known of the trio. One assumes that spacetime has five dimensions, but fields are independent of the fifth dimension (hence the use of the term "cylindrical geometry"). The four mixed components of the metric are identified with the electromagnetic potential. As in Weyl's theory this interpretation is supported by a geometrical interpretation of gauge transformations: in this case they correspond to a particular type of coordinate transformations on the five dimensional space, preserving the cylindrical structure. Further, one sees that the Hilbert action for five dimensional general relativity, when decomposed into its 4+1 components, gives rise to general relativity in four dimensions coupled to Maxwell theory.

In 1926 Oskar Klein proposed that the fifth dimension has the topology of a circle rather than a real line (as implicit in Kaluza's paper).

In the sixties Kaluza and Kleins’s five dimensional world was generalized to the nonabelian case, see

B. de Witt, Dynamical theory of groups and fields, in "Relativity, groups and topology", Gordon and Breach, NY , 1964

R. Kerner, Ann. Inst. Poincare 9, 143, 1968
In modern language, these theories are just the Riemannian geometry of a principal bundle. Given a metric in the base space M, an invariant metric on a group G and a G-connection on M, one constructs a metric in the principal G-bundle over M where the connection is defined.

While these papers describe a unified theory, they do not explain why the world should have the specific structure that is being postulated. In the seventies and eighties people looked for a mechanism that explains this "spontaneous compactification". Except for the five dimensional case, this requires the presence of matter in the higher dimensional spacetime, with an energy-momentum tensor having nonvanishing components in the extra dimensions. Examples of such solutions are given here and here. It proved very difficult to construct realistic models along these lines, see e.g. here. In such models the existence of gauge fields is often postulated in the higher dimensional world, betraying the original KK idea of the gauge fields as components of the metric. At some point in the eighties this research morphed into string theory. The unification mechanism implicit in string theory is completely different, but strings require higher dimensions and it was work on KK theories in the seventies and eighties that made the idea of a higher dimensional spacetime widely acceptable.

Einstein - Maier (1931), Einstein – Maier (1932)
This is probably the least known of the old approaches to unification. Einstein and Maier assumed that spacetime is four dimensional but the "tangent space" at each point is five dimensional. Then, the connection is an object with three indices, one of which (the "form index") still ranges from one to four while the other two (which together can be viewed as the "algebra index") range from one to five. When one specializes the algebra indices, the components with values from one to four can be identified with the gravitational connection while the component 5-5 can be identified with the electromagnetic potential. The mixed components (1-5,2-5,3-5,4-5) do not have a direct interpretation. The crucial question here is: how does one define the directions "1-2-3-4" in this five dimensional space? Einstein and Maier simply assume that there is a two-index tensor with one "form" index ranging from one to four and one index in the "extended tangent space", ranging from one to five, and that the 4x5 matrix representing this tensor has rank four. The directions "1-2-3-4" are conventionally identified with the subspace where this matrix is nonzero, while "5" is the orthogonal direction. Introducing such tensors is fully within the tradition of general relativity, but from a modern point of view one could see this as an early example of the notion of "symmetry breaking": the original five dimensional space is fully isotropic, but this isotropy is broken by the presence of the rank-four two-index tensor. If the two-index tensor was zero, there would be no symmetry breaking and one could not tell electromagnetism and gravity apart.
As in the case of Weyl's theory, also here electromagnetism is a gauge theory for the noncompact group R. One could make this into a gauge theory for the noncompact group U(1) by further enlarging the tangent space to six dimensions and identifying the electromagnetic potential with the gauge field for rotations in the 5-6 plane. More generally, the nonabelian version of Einstein-Maier's theory has been given here and here.
Since it involves higher dimensional spaces, Einstein-Maier theory bears some superficial resemblance to the KK approach, but in reality it is the logical opposite: in KK theory "spacetime structures are turned into internal structures" by the assumptions of a cylindrical structure, whereas here "internal structures are turned into spacetime structures" by use of the two-index tensor..
Of these three approaches, it is the one that most closely resembles the concept of unification as used in the standard model and its GUT extensions.