Yulii Khariton was born to journalist Boris Osipovich Khariton and actress Mirra Yakovlevna Burovskaya, a Jewish family. His father worked for the newspaper Rech, the main organ of the Constitutional Democratic Party. In 1922, by Lenin's decree, the elder Khariton was expelled from Soviet Russia on one of the so-called Philosophers' ships, subsequently working for an emigrant newspaper in Latvia. After the annexation of Latvia by the Soviet Union, Boris Khariton was arrested by the NKVD and died in GULAG. Yulii's mother, Mira Burovskaya, was also an emigre and in the 1930s joined the Zionist immigration to the British colony of Palestine. Yulii was forbidden to contact his parents after he had started classified work.

Khariton studied at the Leningrad Polytechnical Institute (1920 – 1925) under Abram Ioffe and then at the University of Cambridge (1926 – 1928) under Ernest Rutherford, where he received a doctor's degree. From 1931 – 1946 he was head of the Explosion Laboratory at the Institute of Chemical Physics. In 1935 he received his doctorate in physical and mathematical sciences. During this period, Yulii Khariton and Yakov Zel'dovich conducted experiments regarding chain reactions of uranium. He was elected as a corresponding member of the USSR Academy of Sciences in 1946, and as a full member in 1953. He received the Hero of Socialist Labor award in 1949, 1951, and 1954; an Order of Lenin in 1956; and three Stalin Prizes in 1949, 1951, and 1953. He also received a Gold Medal of I.V. Kurchatov in 1974 and a Great Gold Medal of M.V. Lomonosov in 1982.