Rapidly cycling proteins of the early secretory pathway can operate as cargo receptors. Known cargo receptors are abundant proteins, but it remains mysterious why their inactivation leads to rather limited secretion phenotypes. Studies of Surf4, the human orthologue of the yeast cargo receptor Erv29p, now reveal a novel function of cargo receptors. Surf4 was found to interact with endoplasmic reticulum-Golgi intermediate compartment (ERGIC)-53 and p24 proteins. Silencing Surf4 together with ERGIC-53 or silencing the p24 family member p25 induced an identical phenotype characterized by a reduced number of ERGIC clusters and fragmentation of the Golgi apparatus without effect on anterograde transport. Live imaging showed decreased stability of ERGIC clusters after knockdown of p25. Silencing of Surf4/ERGIC-53 or p25 resulted in partial redistribution of coat protein (COP) I but not Golgi matrix proteins to the cytosol and partial resistance of the cis-Golgi to brefeldin A. These findings imply that cargo receptors are essential for maintaining the architecture of ERGIC and Golgi by controlling COP I recruitment.