I know this sounds so silly, but our son can't go to sleep on his own. We literally think he doesn't know how to go to sleep. I know sleep is a learned behavior, but we're at a dead end road.
From the beginning, our son has never been a good sleeper, he slept in the family bed & sleeps great, but he's been in his own bed since 18 months. He will lay for over an hour with his eyes opened. We have to lay with him to get him to sleep. It's not that we mind that, but we are starting to get angry with one another, b/c of the nighttime battle. We have a routine, that ultimately ends up in his room, we've tried soft music, nightlight, reading, singing etc...nothing helps him go to sleep.
The opposite is true in our bed. He'll fall right to sleep in our bed w/ us. I know that it's not good for us to lay with him until he goes to sleep & then leave, b/c when he wakes up, the room is not the same as it was when he fell asleep & therefore, he's not able to put himself back to sleep.
It's becoming very unhealty for our family, when I go to sleep angry every night, b/c I'm at my wits ends, & dad is just as irritated, but he'll lay there after I've already left b/c I've had enough. It doesn't matter what time our son goes down, 8, 8:30, 9 even 10PM, he'll lay there for over an hour. I've read books on how to get them on a healthy sleep pattern. Nothing seems to work. I've tried letting him cry it out, he'll cry for hours & not even pass out from exhaustion.
I need some suggestions b/f it gets even worse. We have an awesome kid, he plays well, he talks & understands more than what his age says is normal, he naps well, he just can't get to sleep on his own. And we're exhausted.
It's possible that he might transition to his own room easier if you keep him company. Taking him out of the family bed and then putting him in his room alone might be too many steps at once. I would consider putting a mattress or sleeping bag on the floor of his room, next to his bed for a week and have one parent lay down separately from him, not "with" him in his bed. Explain that "everyone is going to bed" now and turn down the light. For the first night or two, stay in the room the whole night, then after a couple of successful nights, only stay until he falls asleep. Eventually he will get used to falling asleep in his room at the same time every night and will be able to do it by himself.
Maybe some warm milk or a pre bed bath would help as well.
Sleeping habits from baby-age on is very important for the whole family. We wanted nothing more than to cuddle with our baby in our own bed but we knew that there would come a time where it wouldn't be suitable for any of us so we never went down that road. We spent plenty of time with them laying on the couch and got our cuddle time in there. Then when it's time for bed it's time for bed.
To help our daughter transition from a crib to a bed, we were worried that she wouldn't stay in her bed at night, we bought her really cute bedding and made it special. You have to make sure the time is right to put a kid to bed too. You can't expect them to go to sleep at 7:00 maybe.
We read her a story at bedtime and then left the room. She went to sleep. It is just that easy once you start out that way. When I see some families struggle with the kids in their room it just makes me that much happier that we never went through that. The kids were happier too - they got lots of sleep.
Good luck on breaking the habit but if you work at it you can do it. It is best for the whole family as you probably already know.