Carmichael moved to Toronto in 1911, from his hometown in Orillia, Ontario, to study at the Central Technical School and the Ontario College of Art. He traveled to Antwerp to study painting in 1913, but returned to Ontario in 1914.

Carmichael worked at the commercial art firm Grip Ltd., where he met Tom Thomson and Group of Seven artists Arthur Lismer and J.E.H. MacDonald. After moving to the commercial art firm Rous and Mann, Carmichael met Frederick Varley. Carmichael became a member of the Group of Seven, sharing their passion for painting the raw beauty of the Canadian landscape.

Carmichael frequently visited regions in Ontario such as Georgian Bay and the North Shore of Lake Superior, which became the subject of his landscape sketches.

In the early 1930s, Carmichael became Head of Graphic and Commercial Art at the Ontario College of Art.

He died in Toronto in 1945.