can somebody help me... i should ask pirate john to join the party. but…

can somebody help me...
i should ask pirate john to join the party. but i rejected him before (before i got the dragon bridle). and now, after i got the bridle, i visited him at the jail. but 'rhen' keep rejecting him.
should i do something before visiting john in the jail?
thank you