albatross around one's neck

[albatross around one's neck] {n. phr.}, {literary} Guilt, the haunting past, an unforgettable problem.

Even though it was an accident, John's father's death has been an albatross around John's neck.

Compare: MONKEY ON ONE'S BACK.