Out of state custody question

Justtee33

Hi ladies, I don't know much at all about custody. I'm 13 weeks pregnant, the father of the baby is my ex. We met and lived in California, dated for 3 years, broke up months ago but had a few flings and ended up pregnant. I found out I was pregnant 6 weeks before my planned move back "home" to the east coast. I decided to continue with the pregnancy even though I was moving across the country. Plus we weren't "together" any way. 

 My question is could he ask for some kind of custody? He has 6 kids, one of his ex wives is taking him to court as we speak for child support, he's not working, has no car and lives with his new girlfriend who pays for everything. He's never asked for custody from his other kids mothers. They mostly lived in his area. Even when one moved to Ohio he didn't seem to fight it. 

 I'm wondering if he could file for some way to make me have to pay for him to visit the baby or make it difficult for me to stay on the east coast. Also, I don't want to start putting things in writing or talking about it with him because I don't want to make him mad. He's upset that I followed through with my move and that I'm taking the baby with me. I would like to believe he won't do anything if we just maintain some open communication. I'm willing to bring the baby to visit and he's welcome to visit any time he wants. I'm also, skyping, pictures, updates, phone calls.  But...should I be starting to legally protect myself?? TIA