Actually, the fact that it's a tough business should worry you. In 2011, only 2,338 members of the Writers Guild of America West reported that they made money that year, down 2.3% from the previous year. Given how many people are trying to be writers and how many people have once sold something, that's a tiny, tiny number. It's even a smaller number of writers who can make an entire living on screenwriting alone.
What it boils down to is this: if you're one of the few people who makes it as a screenwriter, then yes, you will have the time for a family (and perhaps more importantly, the money for child care). Feature writing is probably the most flexible, since writing can be done anytime at home, and meetings can be moved around.
On TV, If you're a showrunner, you make the rules, and can bring your kids along. From The New York Times' 2013 profile of SCANDAL creator Shonda Rhimes:
As part of her Shondaland production company, Rhimes oversees some 550 actors, writers, crew members and producers, and her days are optimized to do so. In the morning, she gets her older daughter, Harper, who is 10, off to school and then contends with whatever is most urgent: writing, giving notes on a script and watching casting videos. The televisions in her office and home are connected to a system that allows her to watch real-time editing by her editors. Both of her daughters have rooms across the hall from her office at work. The younger, a perfectly chubby-cheeked 1-year-old named Emerson, comes in every day, clambering onto Rhimes's lap during meetings.Unfortunately, most writers - even pro ones - will never reach Shonda's level. If you're not the boss, it's tougher. You can't just bring your kids into the writers room, and although I've heard of some showrunners letting their writing staffs go at 6 pm, others make them stay until midnight. It just depends on the showrunner and the show. One upside is that TV writing gigs aren't 52-week-a-year jobs; you'll have a hiatus at some point. One downside is that even if you are talented and lucky enough to sell a movie or work on a show, there's no guarantee you'll work again the following year. It might help to have a spouse with a more stable career.
What's probably a more important and relevant consideration for you is that you won't be a full-time working writer (in features or TV) for a long time, if ever. In the meantime, if you try to make connections and work your way up by getting a job as a production assistant, showrunner assistant, script coordinator, etc., you might have to work long hours, and you won't have the authority to make your job flexible. Even writers assistant jobs, which are coveted and competitive, require you to work for hours typing up notes after the writing staff goes home. Also, many lower-level industry jobs also pay very little, meaning that if you have kids, most or all of your salary could go to childcare. Plus, you'll need to spend a lot of time outside of work writing samples on spec.
Plenty of professional screenwriters have families and social lives. It's getting to that level of professional writer that's the problem.