Long-term depression (LTD) underlies learning and memory in various brain regions. Although postsynaptic AMPA receptor trafficking mediates LTD, its underlying molecular mechanisms remain largely unclear. Here we show that stargazin, a transmembrane AMPA receptor regulatory protein, forms a ternary complex with adaptor proteins AP-2 and AP-3A in hippocampal neurons, depending on its phosphorylation state. Inhibiting the stargazin-AP-2 interaction disrupts NMDA-induced AMPA receptor endocytosis, and inhibiting that of stargazin-AP-3A abrogates the late endosomal/lysosomal trafficking of AMPA receptors, thereby upregulating receptor recycling to the cell surface. Similarly, stargazin's interaction with AP-2 or AP-3A is necessary for low-frequency stimulus-evoked LTD in CA1 hippocampal neurons. Thus, stargazin has a crucial role in NMDA-dependent LTD by regulating two trafficking pathways of AMPA receptors--transport from the cell surface to early endosomes and from early endosomes to late endosomes/lysosomes--through its sequential binding to AP-2 and AP-3A.