In western Sydney, local leaders are referring to a "common enemy".  But it's not a shallow rejection of asylum seekers or other migrants, it's a resentment of the other side of town.

Already home to one in 10 Australians, the city's west is getting ready to fight the eastern suburbs and north shore for its share of trains, roads and jobs.  "There is a common enemy, they live in Woollahra and Mosman," Christopher Brown, from a tourism and transport forum, said in reference to ritzy Sydney suburbs.

"We should stand up ... and say we are not going to take it any more," he said, according to The Daily Telegraph.

What's it like where you live?  Tell us using the form below.

Mr Brown was speaking at a population summit - in the city's west - which was also attended by Prime Minister Julia Gillard.  Population is shaping as one of the defining election campaign policies.

When it was her turn to speak, she said it was time to rethink whether more and more people could be sent to western Sydney and southeast Queensland, also home to marginal seats crucial to Labor's re-election chances, or whether those areas were full.

Speaking on radio this morning, she has denied she is putting up the "house full" sign, but instead has said she just wants to make sure the nation gets the right people going to the right places.

"I don't want to look at Australia and leave undisturbed a situation where in some parts of the country we've got high youth unemployment -more than 10 per cent, 15 per cent, sometimes 20 per cent - and in other parts of the country we are crying out for skilled labour," she said.

The summit heard about gruelling commutes from suburban population centres into CBDs and a lack of infrastructure in the areas where most people actually live. 

"The government is about to spend $40 billion on an internet network.  What are the benefits, aside from faster porn and pirate movies?" said the Australia Institute's Richard Denniss.  "They say it's going to come in handy, but if you want to build some trains, that's going to be (too expensive).

"I will know we are ready for a big population when I see spare seats on the train, when hospitals are not stretched to capacity."

Ms Gillard told the audience it was time to "take a breath" on population policy, something she said soon after ousting Kevin Rudd to become prime minister. 

Mr Rudd had said he was a "fan" of the idea of "Big Australia" in which forecasts put the national population at 35 million in the next couple of decades.  Ms Gillard has said we should not "hurtle towards" a target like that.

"Surely it is time for governments to ask this question: Can we really ask western Sydney to keep absorbing hundreds and hundreds of thousands of people without regard for the key issue of quality of life," she has said.

The same debate is going on in other growth areas, including southeast Queensland where the state government is planning three new centres to be developed between Brisbane and the Gold Coast.

Opposition Leader Tony Abbott has also rejected a push to populate.  He has said he doubted the "Big Australia" push months ago and Ms Gillard has only caught up recently to improve her chances at the federal election.

But his predecessor and current eastern Sydney MP, Malcolm Turnbull, told the summit Ms Gillard's comments showed her "moving forward" slogan was rubbish.  He has said growth is inevitable and a good government should be able to manage it properly.

Business groups want more skilled migration to fill skills shortages around the country in certain industries.  Woodside and National Australia Bank chairman Michael Chaney has warned that "paranoia" on asylum seekers is spoiling a sensible debate.

"We need a growing population to fill jobs in our growing economy," he said.  AGL Energy chairman Mark Johnson said: "If continued economic growth is a goal of ours, population growth should be part of that."

- original reporting by Vikki Campion