The Office of Fair Trading (OFT) has asked the Competition Commission to investigate the market for aggregates, cement and ready-mix concrete.
It said five providers account for 90% of the cement market, 75% of aggregates and 68% of ready-mix concrete.
The OFT said it was worried a lack of competition had raised building costs, meaning the government was paying too much for schools, hospitals and roads.
Sales of concrete, cement and aggregates were worth £3.4bn in 2009.
"We are concerned that competition is not working well in these sectors," OFT chief executive John Fingleton said.
Tarmac-owner Anglo American and France's Lafarge recently announced plans to combine their businesses in the UK, further reducing the number of providers.
Hanson UK, Cemex and Aggregate Industries are the other big players in the industry.
The OFT is worried about the way in which the companies work together, citing major firms supplying each other with aggregates and cement as well as "engaging in joint-ventures and asset swaps".
It also said it had received complaints about raw materials providers charging more to independent construction companies than they do to their own affiliates.