Ankyrin repeat family A protein 2 (ANKRA2) interacts with the plasma membrane receptor megalin and the class IIa histone deacetylases HDAC4 and HDAC5. We report that the ankyrin repeat domains of ANKRA2 and its close paralog regulatory factor X-associated ankyrin-containing protein (RFXANK) recognize a PxLPxI/L motif found in diverse binding proteins, including HDAC4, HDAC5, HDAC9, megalin, and regulatory factor X, 5 (RFX5). Crystal structures of the ankyrin repeat domain of ANKRA2 in complex with its binding peptides revealed that each of the middle three ankyrin repeats of ANKRA2 recognizes a residue from the PxLPxI/L motif in a tumbler-lock binding mode, with ANKRA2 acting as the lock and the linear binding motif serving as the key. Structural analysis showed that three disease-causing mutations in RFXANK affect residues that are critical for binding to RFX5. These results suggest a fundamental principle of longitudinal recognition of linear sequences by a repeat-type domain. In addition, phosphorylation of serine 350, a residue embedded within the PxLPxI/L motif of HDAC4, impaired the binding of ANKRA2 but generated a high-affinity docking site for 14-3-3 proteins, which may help sequester this HDAC in the cytoplasm. Thus, the binding preference of the PxLPxI/L motif is signal-dependent. Furthermore, proteome-wide screening suggested that a similar phosphorylation-dependent switch may operate in other pathways. Together, our findings uncover a previously uncharacterized sequence- and signal-dependent peptide recognition mode for a repeat-type protein domain.