First off, where are you looking? Children will always appear on the parent's record regardless of if the children are alive or dead, living in the ward or not. Many clerks seems to confuse the "genealogical" information found on a individual record with the household information.
So I'm assuming that if you go into "individual" that you can see these children:
If they are truly non-member, you can just delete them. But if they are "children of record", (created before age 8) you need to see Handbook 1: 13.6.2.
Have you searched the
Help Center? Try doing a Google search and adding "site:churchofjesuschrist.org/help" to the search criteria.
So we can better help you, please edit your
Profile to include your general location.