Divergent transcription occurs at the majority of RNA polymerase II (RNAPII) promoters in mouse embryonic stem cells (mESCs), and this activity correlates with CpG islands. Here we report the characterization of upstream antisense transcription in regions encoding transcription start site associated RNAs (TSSa-RNAs) at four divergent CpG island promoters: Isg20l1, Tcea1, Txn1, and Sf3b1. We find that upstream antisense RNAs (uaRNAs) have distinct capped 5' termini and heterogeneous nonpolyadenylated 3' ends. uaRNAs are short-lived with average half-lives of 18 minutes and are present at 1-4 copies per cell, approximately one RNA per DNA template. Exosome depletion stabilizes uaRNAs. These uaRNAs are probably initiation products because their capped termini correlate with peaks of paused RNAPII. The pausing factors NELF and DSIF are associated with these antisense polymerases and their sense partners. Knockdown of either NELF or DSIF results in an increase in the levels of uaRNAs. Consistent with P-TEFb controlling release from pausing, treatment with its inhibitor, flavopiridol, decreases uaRNA and nascent mRNA transcripts with similar kinetics. Finally, Isg20l1 induction reveals equivalent increases in transcriptional activity in sense and antisense directions. Together these data show divergent polymerases are regulated after P-TEFb recruitment with uaRNA levels controlled by the exosome.