Human inactive X chromosome (Xi) forms a compact structure called the Barr body, which is enriched in repressive histone modifications such as trimethylation of histone H3 Lys9 (H3K9me3) and Lys27 (H3K27me3). These two histone marks are distributed in distinct domains, and X-inactive specific transcript (XIST) preferentially colocalizes with H3K27me3 domains. Here we show that Xi compaction requires HBiX1, a heterochromatin protein 1 (HP1)-binding protein, and structural maintenance of chromosomes hinge domain-containing protein 1 (SMCHD1), both of which are enriched throughout the Xi chromosome. HBiX1 localization to H3K9me3 and XIST-associated H3K27me3 (XIST-H3K27me3) domains was mediated through interactions with HP1 and SMCHD1, respectively. Furthermore, HBiX1 was required for SMCHD1 localization to H3K9me3 domains. Depletion of HBiX1 or SMCHD1, but not Polycomb repressive complex 2 (PRC2), resulted in Xi decompaction, similarly to XIST depletion. Thus, the molecular network involving HBiX1 and SMCHD1 links the H3K9me3 and XIST-H3K27me3 domains to organize the compact Xi structure.