im at paradise falls. im at the part where im waiting for bandits to co…

im at paradise falls. im at the part where im waiting for bandits to come. it says after youve built all these buildings. well the mines are all gone. does that matter? so you can see how long ive been at this. i ve had only 4 bandits come to fight. im assuming theres more. when do they come?