But officials in the oil-rich state are confident a set of new measures will ensure the annual Muslim rite passes without incident.
The main concern is the risk of attacks by militants bent on destabilising the royal family.
Of special concern is al Qaeda, which has killed about 170 people during its 20-month campaign of violence in the kingdom.
In December gunmen attacked the US consulate in Jeddah, and suicide bombers have targeted government buildings in the capital Riyadh.
Saudi authorities say 50,000 men, including 7,000 Special Forces, have been mobilised in the holy hajj sites and the nearby Red Sea port city of Jeddah.
“We are sure that everyone coming here is doing so to perform hajj, but we strongly say ‘no’ to anyone who wants to disturb security” said Interior Minister Prince Nayef bin Abdul-Aziz.
He also said security forces would be doing all they could to stop militants using hajj as a staging post for crossing into neighbouring Iraq.
A duty for every able-bodied Muslim at least once in a lifetime, the hajj includes ritual circling of the Kaaba, a cube-shaped structure in the Grand Mosque where Islam says God’s presence is most felt on Earth.
Over one and a half million foreign pilgrims are expected to flood into the birthplace of Prophet Mohammed to take part, in addition to the half a million from within the kingdom.
And hundreds of thousands more could stream in from the region around Mecca at the hajj climax at Mount Arafat Wednesday.
“There could be up to three million people on that day,” said Prince Nayef, adding Saudi Arabia had been unfairly criticised over past stampedes.
“The number of people who died in the last five years is a tiny percent of all the pilgrims, maybe less than the percent who die in London traffic,” he said.
In 1990, 1,426 pilgrims died in a stampede and in 1997, 343 pilgrims were killed by a fire in a pilgrims’ camp.
Last year more than 250 pilgrims were killed in a stampede at the climax of last year’s hajj as thousands surged to throw stones at pillars symbolising the devil.
Authorities say they’ve improved the flow of pilgrims into the Jamarat area.
New measures include signs warning of overcrowding, extra emergency exits and a special unit to bulldoze makeshift roadside camps, which the government blamed for last year’s deadly crush.
This year, pilgrims are due to travel to Mina on Tuesday evening where they will spend the night under canvas before catching the bus or walking to Mount Arafat for a ritual that symbolises the Last Judgement.
They will return to Mecca on Thursday, the first day of Eid al-Adha or Feast of the Sacrifice, when most will sacrifice an animal, generally a sheep, in remembrance of Abraham’s readiness to sacrifice his son to God.
To cater for demand, Saudi shopkeepers have imported more than one million sheep, largely from Sudan and Uruguay.
On Friday and Saturday, the pilgrims will return to Mina to stone the three statues representing Satan, the ultimate rite of pilgrimage.
Waiting for the rites to begin, pilgrims have spent most of their time praying in Mecca’s Grand Mosque, the holiest in Islam, or walking around the Kaaba, the cube-shaped structure revered by Muslims as the house of God.