The retinal ganglion cells (RGCs) are the sole output neurons in the retina that form the optic nerve and convey light signals detected by photoreceptors to the higher visual system. Their degeneration and damage caused by glaucoma and injury can lead to blindness. During retinogenesis, RGCs are specified from a population of multipotential precursors capable of generating RGC, amacrine, horizontal, and cone cells. How the RGC fate is selected from these multiple neuron fates is unknown at present. Here we show that the previously unsuspected POU domain transcription factor Brn3b (brain-specific homeobox/POU domain protein 3b) plays such a critical role. Loss of Brn3b function in mice leads to misspecification of early RGC precursors as late-born RGC, amacrine, and horizontal cells, whereas misexpressed Brn3b suppresses non-RGC cell fates but promotes the RGC fate. Microarray profiling and other molecular analyses reveal that, in RGC precursors, Brn3b normally represses the expression of a network of retinogenic factor genes involved in fate commitment and differentiation of late-born RGC, amacrine, horizontal, and cone cells. Our data suggest that Brn3b specifies the RGC fate from multipotential precursors not only by promoting RGC differentiation but also by suppressing non-RGC differentiation programs as a safeguard mechanism.