He was originally cast as "Michael" in The Deer Hunter (1978), as the second movie of a three movie deal with Universal Studios. Because he did not believe that the character would travel around the world to find his friend, he quit the picture. Universal executives were furious, but they agreed to let him out of his Universal contract if he made Jaws 2 (1978), which he did. He later regarded pulling out of The Deer Hunter as the career decision he most regrets.
It's one thing to get rejected for a part. Kevin Costner is still bitter about not getting to play H.I. in Raising Arizona. It's a little worse to turn down a part that you'd be perfect for. Michael Madsen turned down the role of Vincent Vega in Pulp Fiction, which Tarantino wrote with Madsen in mind. But getting the gig and then walking away from a classic movie because of "creative differences"? You don't think a troubled vet would return to Vietnam to bring his buddy home, but you're fine with a shark terrorizing resort islanders? That's gotta hurt.
By the way, there's a fun site with info on which movies actors turned down or were rejected for famous roles. Here are my favorites:
* John Denver turned down the lead role in An Officer and a Gentleman because he didn't like the script. "I got nowhere else to go!"
* Jack Nicholson turned down the role of Michael Corleone. The studio wanted Robert Redford or Burt Reynolds to play Michael Corleone. Even after filming started, there was pressure to replace Al Pacino. Luckily, once execs saw the dailies for the scene in which Michael guns down Sollozzo and the cop (warning: spoiler alert!), they backed off.
* Charlton Heston was considered for the role of Chief Brody in Jaws. "We're going to need a bigger gun!"
* O.J. Simpson was passed over for the role of the terminator because producers feared he wouldn't be taken seriously as a killer. I can picture him chasing down Sarah Connor in a golf cart.
* Clint Eastwood, Dustin Hoffman, Steve McQueen, Nick Nolte, Al Pacino, and John Travolta were all considered for the part of John Rambo in First Blood. That movie could have been much better -- or a whole lot worse.
* Lucille Ball was considered for the role of Scarlett O'Hara in Gone with the Wind. "Oh Ashley, Ashley. Whaaah!"