The departure of Disney chairman Dick Cook is the main reason the star is having second thoughts about revisiting the role for a third time.
The star called the Los Angeles Times to say he is not as keen to appear in the film with Cook no longer being part of the company.
Depp said: "There's a fissure, a crack in my enthusiasm at the moment... it was all born in that office."
Referring to Cook, the actor revealed: "He said, 'We're thinking about doing this Pirates Of The Caribbean movie,' I said I was in. This was before there was a script or anything.
"When things went a little sideways on the first Pirates movie and others at the studio were less than enthusiastic about my interpretation of the character, Dick was there from the first moment. He trusted me."
He added that while some Disney executives were "opposed to my body language," Cook "was completely supportive of me."
Cook and Depp - in full Jack Sparrow costume - announced the new title for Pirates four, Pirates Of The Caribbean: On Stranger Tides last week, and revealed part of the plot that Captain Jack will set off in search of the Fountain Of Youth.