Located approximately halfway between Cape Town and Johannesburg, Colesberg provides a convenient stopover for travellers between these two destinations.. The town lies in typical Karoo veld and is surrounded by koppies (little hills). The most famous is Coleskop, which can be seen from a distance of over 40km. Early travellers called it "Towerberg" ("Magic Mountain"). The curious thing about this koppie is that as you travel towards it, it never seems to get nearer.

The first people to inhabit the Colesberg district were stone-age hunter-gatherers. They were followed in the early 19th century by "trekboere", migrant farmers and missionaries. By 1814, a mission station had been established in the hopes of bringing peace to what was an extremely unruly frontier area. By 1820 several huge farms had been established in the district and in 1822 the farmers petitioned for the establishment of a town. The town was eventually founded in 1830 and named after Sir Lowry Cole, the Cape Governor.

Colesberg has a rich history closely linked with to the legendary characters of South Africa's diamond industry. John O'Reiley, who purchased the first diamond found in South Africa from its owner, Schalk van Niekerk, took it to Colesberg for testing. It was used to scratch "DP", the initials of Draper and Plewman, a store which still exists, on the shop's window.