new FamilySearch merging

Perhaps others of you in the beta have run into this issue and everyone might have some ideas - In the beta I uploaded a GEDCOM which has been my research on one particular line for the last year or so. All of the data except the last couple dozen records I've found in the last few visits to the FHC has been submitted to TempleReady with most of the work completed by my extended family. When I uploaded, it did not recognize that I am the same submitter.

Does anyone on the "inside" know if we'll be able to link our profile to the single or multiple submitter profiles from our submissions under the old system? I don't really care to merge everyone from my upload with everyone from my old submission only because it can't recognize that the name/address on my former submission are the same as my profile now. Those who have an inkling at the architecture or the issues involved, how difficult do you think it would be to merge submitters in addition to individuals?

That is a very good point. If someone has submitted something in the past and they now use a different username or have a different address, they also would not be allowed to edit their former submissions.

I do remember reading somewhere that there will be a way to inherit a submission from a deceased individual, so there may be some method to claim your former submissions. It would make sense to be able to merge all your different submission records to yourself so others could see your current contact information. An old submission address or expired e-mail address does no one any good.

I recall seeing instructions somewhere in the help popups that a current contributor can send in something to claim ownership of his/her own earlier submissions. I think it said you could choose to claim ownership of a single contribution for a particular ancestor vs. everything made under an earlier (old system) contributor ID.

Yes, we do need something that is updatable when it comes to submitter information. Not like the old Ancestral File where all that stuff is still there sometimes 30 years after that began.

The 'new FamilySearch', based on those screenshots at the link posted elsewhere on this board yesterday, is the promised blending of the two data management techniques that were used in AF and the PRF. You'll be able to see all submissions, but in 'your' view, your submission in your tree will be the 'top' submission, but all the rest will still be available for view by everyone. You''ll be able for the first time to actually compare data.

And that is why the 'dispute' function. Maybe 'challenge' is a better word, because dispute connotates an negative argument, but maybe a word like 'challenge' or one of its synonyms, would be a more appropriate word to call that feature. The 'dispute' function allows for comparisons of differing data to be highlighted among two or more submitters, and they can then share reasons why they think the data they have is what it is.

There should be a feature at least in the immediate future to do address changes, email address changes, etc., so that contact can be maintained. Dealing with the deceased submitter issue is another matter that will take more work to resolve properly.

JamesAnderson wrote: The 'new FamilySearch', based on those screenshots at the link posted elsewhere on this board yesterday, is the promised blending of the two data management techniques that were used in AF and the PRF. You'll be able to see all submissions, but in 'your' view, your submission in your tree will be the 'top' submission, but all the rest will still be available for view by everyone. You''ll be able for the first time to actually compare data.

I am not sure if this is a correct statement. I asked the Question in another thread if when you go to preferences and select to show "your" own submissions if other data is also included. The answer from Roger was no, in that mode you only see your information and no one elses.

I don't believe that in the standard view, where all submitters information is available, that your information will always be listed at the top. I think that information with sources is listed at the top ahead of other info, so if your submission doesn't have soruces it may not be the default information. Maybe some of the Beta testers could let us know if their info is always listed first in the default main view.

Would there be a way to accept others changes as your own and easily have them added in to your own file?

Currently I work as a "coordinator" for my family. I have several different family members working on genealogy and periodically they send their PAF files back to me, I merge in their changes, and then send them back a copy of the current complete PAF file (which will have changes from other family members included as well). I'll also review records to see if it looks like the content is pretty solid (good source records if available, information well checked, etc.), and then make sure it has been manually verified against the familysearch website for ordinance information (the temple ready "auto" check thing just doesn't do it justice). Once those steps are complete I'll give it to someone to do the initial submission to one of the temples and start the ordinance work moving (they may only do part of the ordinances and then pass the cards to someone else).

At this stage, I'm trying to figure out how all of this fits in with the new website and how much of this it might make easier for me to do. At the least, I'll need for family members to easily be able to share information with each other. We don't need or want everyone running separate files/info at this point. If there's a dispute we tend to iron it out in-house It would be nice though to have a clear picture of what changes are coming over from someone though. Even though person X may be working on branch Y, it's not uncommon for branch Y to have some piece that suddenly forks into branch Z that someone else was working on (typically side lines, my tree isn't all that vertical ).

If I understand things correctly from the training materials in the beta, the new system intends to put you out of a job, as coordinator, that is. The idea with the new system seems to be that there won't be anyone's "file" any more, but everyone will make incremental changes and additions to the big database. Anyone else can go back through the new data and check it.

That said, if a family wanted to continue to funnel things through a single coordinator, the new system would appear to handle that as if the coordinator were doing all the research himself/herself.

Right now, there is no capability to download data as GEDCOM or other file format. However, there is an API planned that should support local desktop genealogy application programs, and that should enable download and conversion to GEDCOM if somebody writes the code to do that. Uploading GEDCOMs is supported in the new system.

HTH

This new web site will be a public site. There will not be MY file or YOUR file, it will be the "new FamilySearch" file. All temple work will be handled through this file, there will be no other way to clear names for the Temple. Everyone will use it, one big happy world family. You will have the ability to put a hold on Temple Ordinances for names you submit. This will allow you as a family represenative to decide who gets to do the ordinances. But if some distant relative extends a line and adds names to a family line that is also your family line, they will have the right to do the same to the names they have added. It will be much easier to correlate the work between people because the main file will be online and available to everyone. You will be able to correlate with everyone working on your lines, not just those close relatives you know. Anyone who has submitted to a family name can update their own submission. If you have not submitted to a name, you will only be able to make comments about other peoples submissions, you will not be able to change what they have submitted. If someone has sent in something you want, you can add that to your submitted record. In a later update to the site you will be able to download from the system but for now you would have to copy and paste. I have read that an updated PAF or a replacement for PAF will allow syncronizing with the New FamilySearch. Through the API other web sites and software programs will have access to the information as well.

Would there be an easy way to merge someone else's entire tree with your own without having to approve the merge of completely identical records? I don't really want our trees disintegrating into a bunch of fragmented pieces instead of one nice whole.

The downloading part is still going to be fairly critical for us too as sometimes you need an offline copy of the whole tree when you're in remote places doing research.

Not sure how much I like the everything being public too. The quality of data in my PAF file varies greatly. Some of it I don't think could be any more solid, other data though is closer to "research notes" than it is something I'd consider even half way accurate. I guess I wouldn't want someone using my possible "trash" data, or marking it up with a bunch of notes about how it's wrong (which wouldn't be a surprise - however pointers to good data would obviously be appreciated, but that could just as easily be found if they saw the missing branch on my tree instead of some questionable data).

gblack wrote:Would there be an easy way to merge someone else's entire tree with your own without having to approve the merge of completely identical records? I don't really want our trees disintegrating into a bunch of fragmented pieces instead of one nice whole.

The downloading part is still going to be fairly critical for us too as sometimes you need an offline copy of the whole tree when you're in remote places doing research.

I don't know all the details about automatic merging when uploading whole files, but at least some of the process _is_ automatic. I uploaded a GEDCOM file with 8 people in it. The system reports that all 8 were added as new individuals, but I don't believe I had to manually merge all 8. Then, I uploaded the same GEDCOM file again but with more sources and notes, and it automatically merged in 5 of the 8 people. The other three were trivial to merge, if anything actually needed to be done.

Does that help at all?

gblack wrote:Would there be an easy way to merge someone else's entire tree with your own without having to approve the merge of completely identical records? I don't really want our trees disintegrating into a bunch of fragmented pieces instead of one nice whole.

The downloading part is still going to be fairly critical for us too as sometimes you need an offline copy of the whole tree when you're in remote places doing research.

I don't believe downloading from new FamilySearch (NFS) will be part of the original release, however the current FamilySearch will be running at the same time and some downloading can be done there. New submissions will be the only info not available to download. When NFS is released it will consist of Church Membership records, Temple Ordinance Index (IGI), Ancestral File (The old Four Generation program), Pedigree Resource File, and any other indexes that are now part of FamilySearch. You can download from most of these databases either at the Family History Centers (FHC) or online.

All these databases will have the names linked together in NFS. There is a lot of undocumented information in some of these files. The hope is to get everyone working together to document and clean up this information and prevent so much duplication of effort.

The only files that I know of that can be merged in PAF automatically are identical records with the same Ancestral File Number (AFN), so if you use the merge by AFN option, PAF can auto merge those records. When you download someone else's entire database you can compare that file with yours side by side, record by record and just select the information you want to move to your record using "PAF Insight", this program is available at most FHC's to use and is available to purchase online. Do a Google search for "PAF Insight" for more info.

When people come into the FHC the first step in helping people get started doing family history work is to get together what their family has done and what the Church already has. So a lot of people are spending hours downloading and matching and merging to get to a starting point. The next step I do is have people start verifying the information that they do not have sources for, so a lot of people are having to relocate sources because most of the databases do not include the source citations, a lot of time can be saved once this process is moved online and you can see sources that someone has already docummented. Unfortunatly a lot of people skip this step and start running the downloaded names through TempleReady and redoing work that has been done since TempleReady was last updated (2000). In our FHC all the workers know to check all Temple submissions against the online IGI, which is up to date, but there are a lot of FHC's in the world and a lot of people can use the computers and TempleReady without assistance, so a lot of temple work gets repeated. This new system is designed to stop that problem. They stopped updating TempleReady expecting this new system to come online, but it is taking a lot longer to get it working than many expected.

rmrichesjr wrote:I don't know all the details about automatic merging when uploading whole files, but at least some of the process _is_ automatic. I uploaded a GEDCOM file with 8 people in it. The system reports that all 8 were added as new individuals, but I don't believe I had to manually merge all 8. Then, I uploaded the same GEDCOM file again but with more sources and notes, and it automatically merged in 5 of the 8 people. The other three were trivial to merge, if anything actually needed to be done.

Does that help at all?

I believe I read somewhere that unique numbers are assigned to names in new FamilySearch so when you resubmit a name that has that same number it can be combined.

gblack wrote:Not sure how much I like the everything being public too. The quality of data in my PAF file varies greatly. Some of it I don't think could be any more solid, other data though is closer to "research notes" than it is something I'd consider even half way accurate. I guess I wouldn't want someone using my possible "trash" data, or marking it up with a bunch of notes about how it's wrong (which wouldn't be a surprise - however pointers to good data would obviously be appreciated, but that could just as easily be found if they saw the missing branch on my tree instead of some questionable data).

I totaly agree, I do not encourage patrons at our FHC to submit info without sources. You can create a GEDCOM of the good info and send that in. I also have a lot of info in my PAF files that I have downloaded for Ancestral File and other Databases that I have not located sources for yet. I will only be submitting my sourced information. Most of the info I have download will already be part of "new FamilySearch" (NFS) anyway, whether it is accurate or not because Ancestral File and Pedigree Resource File will be linked there.

Our families records need a lot of the sources moved from notes to sources, so we have a lot of work to do to clean up our info. That is the kind of thing that NFS will be good for. I have relatives asking me all the time "What can I do?". When NFS is out, I can tell them to go to the notes look up the sources and get images of the records and enter the citations into Sources and e-mail me the images so we will have them all together and ready to attach to the source citations when that feature is added in a later version. We have family web sites now where we post some of our source images.

Right. I'll confess that at some stages I've been known to export it all to GEDCOM, use Perl to manipulate the GEDCOMs to where I want them, and then reimport. PAF, while good, is still lacking a lot of features that really wouldn't be that difficult to add. PAF does have a unique serial number option itself though.

GarysTurn wrote:I totaly agree, I do not encourage patrons at our FHC to submit info without sources. You can create a GEDCOM of the good info and send that in.

That's kind of the whole problem though, I was hoping to be able to use the online site as my primary means of working on the data and coordinating with family members, while only occasionally exporting to PAF or some such for offline research and then merging back in. Now if want to prevent garbage info in a public repository it sounds like I won't be able to do that.

GarysTurn wrote: Our families records need a lot of the sources moved from notes to sources, so we have a lot of work to do to clean up our info. That is the kind of thing that NFS will be good for. I have relatives asking me all the time "What can I do?". When NFS is out, I can tell them to go to the notes look up the sources and get images of the records and enter the citations into Sources

If there's "your data" and "their data" though without any real way of marking it as some sort of group effort, how do you get all of that into once nice complete file/area/repository/etc. without them using your account to make the modifications?

GarysTurn wrote: and e-mail me the images so we will have them all together and ready to attach to the source citations when that feature is added in a later version. We have family web sites now where we post some of our source images.

I guess I would think when they bring this online you would be able to do specific types of source citations, like for example a citation of the 1880 census would link directly to an online copy of the 1880 census and the particular page in question. If so, then your images will only get in the way there as they'll be more of the "external source, imaged attached" sort of deal (for what it's worth I collect images or copies as the case may be of relevant census pages too ).

rmrichesjr wrote:I don't know all the details about automatic merging when uploading whole files, but at least some of the process _is_ automatic.

PAF has a hidden uniquely assigned number. When importing, if PAF (and presumably the new FS) sees that same unique ID, it knows it's the same individual without having to compare the person's data.

So if your upload came from the same PAF file, that's probably why. But if someone were to submit the same data hand typed into another program, it wouldn't auto merge because it's either missing that unique ID or has a different one.

gblack wrote:If there's "your data" and "their data" though without any real way of marking it as some sort of group effort, how do you get all of that into once nice complete file/area/repository/etc. without them using your account to make the modifications?
.

In the new FamilySearch, anyone can add information about stuff already there. If somebody added a fact of some sort, someone else can add a source to it after the fact. In the beta system, I just added a (bogus) source for a birth event for someone I had not previously contributed any data. I clicked on the 'edit' link for the event, then clicked 'Add Other Opinion' and added the source information. It did not create a new event but now lists my source. Of course, it lists me as a contributor to the birth event too, but that is at least arguably correct.

gblack wrote:That's kind of the whole problem though, I was hoping to be able to use the online site as my primary means of working on the data and coordinating with family members, while only occasionally exporting to PAF or some such for offline research and then merging back in. Now if want to prevent garbage info in a public repository it sounds like I won't be able to do that.

If there's "your data" and "their data" though without any real way of marking it as some sort of group effort, how do you get all of that into once nice complete file/area/repository/etc. without them using your account to make the modifications?

I guess I would think when they bring this online you would be able to do specific types of source citations, like for example a citation of the 1880 census would link directly to an online copy of the 1880 census and the particular page in question. If so, then your images will only get in the way there as they'll be more of the "external source, imaged attached" sort of deal (for what it's worth I collect images or copies as the case may be of relevant census pages too ).

If you submit work in progress you can always remove it if it turns out to be incorrect. You could put in notes that it is work in progress.

Everything is linked together as one name with many contribuitors and is updated as a group effort. Everyone can work on that name and add sources to that name, the sources added will show the person who entered the source as the contributor. You won't have your own work area, anyone can contribute.

It will take up to 7 years to scan and index all the Chruch microfilms, until then links to online images on personal web pages will be helpful. We can add link info in source comments but until web links are available people will have to copy and paste the link into their browser.

GarysTurn wrote:If you submit work in progress you can always remove it if it turns out to be incorrect. You could put in notes that it is work in progress.

I recall one of the training overviews saying that work-in-progress data should be put in the database, and that the whole database is or should be considered a work in progress. (My comment: After all, it's not going to be complete and flawless until probably somewhere close to a thousand years from now. 1/2

rmrichesjr wrote:I recall one of the training overviews saying that work-in-progress data should be put in the database, and that the whole database is or should be considered a work in progress. (My comment: After all, it's not going to be complete and flawless until probably somewhere close to a thousand years from now. 1/2

This whole concept is different than what we have now. It will save a lot of effort. In the old system people were downloading their line from Ancestral File doing some research and making a few changes then uploading the entire file back to Pedigree Resource File. So Pedigree Resource File contains a lot of duplicate information. If someone re-submitted their files every few months the PRF gets big and hard to find the new info. But it was the only way we had of sharing our work with others. This new system will be so much better with everyone working together on the same file. And there are controls in place so that information is not lost or changed by someone who did not submit it.

I wasn't sure that I agreed with the 'public' concerns for posting our family files, but there is a very relevant point made in the post. There are 'states' to our research. When managing our own files, we are likely to have very cryptic notes. Has the design considered the various 'states' of an entry?

I'm also curious as to how duplicate entries are going to be handled at the variable level. Will there be a 'collaborative' element to resolve conflicts? If the collaboration identifies a need for a 'merge' of entries, how will this be facilitated?

Indeed, is there any way to share the design scenarios for review/comment?

The answers are in the questions...I'd like to see what questions were considered.

iknovate wrote:I wasn't sure that I agreed with the 'public' concerns for posting our family files, but there is a very relevant point made in the post. There are 'states' to our research. When managing our own files, we are likely to have very cryptic notes. Has the design considered the various 'states' of an entry?

I'm also curious as to how duplicate entries are going to be handled at the variable level. Will there be a 'collaborative' element to resolve conflicts? If the collaboration identifies a need for a 'merge' of entries, how will this be facilitated?

Indeed, is there any way to share the design scenarios for review/comment?

The answers are in the questions...I'd like to see what questions were considered.

A lot of these questions have been answered in this an other forums. Here is a link to a web site which links to a lot of the other sites discussing newFamilySearch. There are Blogs by beta testers, forums and other resources here about newFamilySearch: http://newfamilyhistory.googlepages.com/home

Well since it's all "one tree" we're working on, and since we seem to just be submitting conflicting info about individuals if we differ, at what point does something constitute a different person vs corrections to a person's record? If the name is listed as Jack Frost, and I feel the name is Jack Frostland, and I have different event dates that differ by only a few days, does that constitute a different person? What if I show a branch with an entirely different set of parents?

How is temple worked handled under this? Does someone get to do the work for Jack Frost and someone else for Jack Frostland? Or is Jack Frost's work considered valid for Jack Frostland?

Can temple work be reserved for this? I submit info on my grandfather and want to do his work - can I reserve it or can anyone do it? If I reserve it and fall over dead without doing the work, and the system is never notified, is there a mechanism to "unlock" it so someone else can do it? How about current work that is marked in progress (and has been that way for years) but is not done that is introduced into the system - will we have problems printing cards to do that work?

gblack wrote:Well since it's all "one tree" we're working on, and since we seem to just be submitting conflicting info about individuals if we differ, at what point does something constitute a different person vs corrections to a person's record? If the name is listed as Jack Frost, and I feel the name is Jack Frostland, and I have different event dates that differ by only a few days, does that constitute a different person? What if I show a branch with an entirely different set of parents?

How is temple worked handled under this? Does someone get to do the work for Jack Frost and someone else for Jack Frostland? Or is Jack Frost's work considered valid for Jack Frostland?

Can temple work be reserved for this? I submit info on my grandfather and want to do his work - can I reserve it or can anyone do it? If I reserve it and fall over dead without doing the work, and the system is never notified, is there a mechanism to "unlock" it so someone else can do it? How about current work that is marked in progress (and has been that way for years) but is not done that is introduced into the system - will we have problems printing cards to do that work?

The program will link all the records it determines are the same person. Some individual will have to make the determination if it is the same person or a different person. The system shows possible matches and patrons decide if they are to be combined.

If two submissions are combined, they can be uncombined if someone determines later that it is a different individual.

When you contribute names to the system it gives you the option to save the names for yourself to do the temple work. You can then do the work yourself or release the names to other family or friends to do. They are working on a method for claiming submissions from people who have died. And they are working on how they will handle names submitted and locked for work but nothing is being done for a specific period of time. I am not aware what the solution is, but I have heard that they are aware of the need for a solution and are working on a procedure.

GarysTurn wrote: The only files that I know of that can be merged in PAF automatically are identical records with the same Ancestral File Number (AFN), so if you use the merge by AFN option, PAF can auto merge those records. When you download someone else's entire database you can compare that file with yours side by side, record by record and just select the information you want to move to your record using "PAF Insight", this program is available at most FHC's to use and is available to purchase online. Do a Google search for "PAF Insight" for more info.

This has probably already been discussed, but I've always wondered about how hard it would be to create a more intelligent merging program: Say that Mary Jones, born in 1901 in Geneva, NY married John Smith, born in 1895 in Topeka, Kansas, and had several children, one of whom was Stephen Smith, born in 1920 in Buffalo, NY. If multiple people upload this family, why couldn't a merging program figure out that these are the same people. The argument I've heard is 'there are hundreds of thousands of Mary Jones, etc.' Sure, but how many were born in 1901 in Geneva, NY, married John Smith, born in 1895 in Topeka, Kansas, and had a son named Stephen, born in 1920 in Buffalo, NY?

I would put the odds of two different Mary Jones matching all that at 1 in a million or less. I think you could have an accurate match, even if you didn't have all the dates and places, especially if you had more children's names.

A second element of an intelligent match, even without the above should be that it should figure out that if I merged child A1 in Family A with Child B1 in Family B, then it's a good guess that I can merge everyone in family A with their matches in family B, assuming they have the same first name and birth dates. In other words, once I identify a match, it could go on it's merry way: Multiple generations could be merged very quickly with this approach, simply by making one match, as parents would be children in the next family, and so on.

Again, this sounds easy on paper, but how hard is that? I would imagine it taking a fair amount of CPU time, and probably couldn't be done in real time. The first suggestion could be like a web crawler, constantly looking for possible merges, and the second one would be a batch process that runs asynchronously.

Even better - the program that looks for possible matches could simply flag the matches to the two tree owners as possible matches, and let them decide.

bcpalmer60 wrote:Sure, but how many were born in 1901 in Geneva, NY, married John Smith, born in 1895 in Topeka, Kansas, and had a son named Stephen, born in 1920 in Buffalo, NY?

I would put the odds of two different Mary Jones matching all that at 1 in a million or less.

I don't know how many people have lived on this Earth total from the beginning of time, but we can at least say 6 billion because that is the current number right now. So using the 6 billion number, 1 in a million odds would mean that you are banking on no more than 6,000 Mary Jones that meet the criteria you specified. Even that is overly safe; maybe 1 in 6 billion odds is better!

bcpalmer60 wrote:This has probably already been discussed, but I've always wondered about how hard it would be to create a more intelligent merging program: Say that Mary Jones, born in 1901 in Geneva, NY married John Smith, born in 1895 in Topeka, Kansas, and had several children, one of whom was Stephen Smith, born in 1920 in Buffalo, NY. If multiple people upload this family, why couldn't a merging program figure out that these are the same people. The argument I've heard is 'there are hundreds of thousands of Mary Jones, etc.' Sure, but how many were born in 1901 in Geneva, NY, married John Smith, born in 1895 in Topeka, Kansas, and had a son named Stephen, born in 1920 in Buffalo, NY?

I would put the odds of two different Mary Jones matching all that at 1 in a million or less. I think you could have an accurate match, even if you didn't have all the dates and places, especially if you had more children's names.

A second element of an intelligent match, even without the above should be that it should figure out that if I merged child A1 in Family A with Child B1 in Family B, then it's a good guess that I can merge everyone in family A with their matches in family B, assuming they have the same first name and birth dates. In other words, once I identify a match, it could go on it's merry way: Multiple generations could be merged very quickly with this approach, simply by making one match, as parents would be children in the next family, and so on.

Again, this sounds easy on paper, but how hard is that? I would imagine it taking a fair amount of CPU time, and probably couldn't be done in real time. The first suggestion could be like a web crawler, constantly looking for possible merges, and the second one would be a batch process that runs asynchronously.

Even better - the program that looks for possible matches could simply flag the matches to the two tree owners as possible matches, and let them decide.

A lot of the software out there does take into consideration the possible matches you mention. That is how match and merge programs work now. The final decision is still left up to a person. What you describe as a web crawler type program constantly searching for matches is kind of what will take place in newFamilySearch. NewFamilySearch will link possible matches for you and you will decide if they are real matches. With the structure of newFamilySearch a true merge is not done but a combining of information, so that no information is lost. There will not be as much of a problem created with merging as there is with current software, because things that are combined in error can be un-combined.

bcpalmer60 wrote:Again, this sounds easy on paper, but how hard is that? I would imagine it taking a fair amount of CPU time, and probably couldn't be done in real time. The first suggestion could be like a web crawler, constantly looking for possible merges, and the second one would be a batch process that runs asynchronously.

As I understand it, and as the current beta of the new FamilySearch is set up, the idea is to have human users manually decide which records to match. It sounds like the intent is to allow inspiration to guide the process.

The current beta system has some level of heuristic of some sort that suggests whether the system thinks the people are probable matches or are not as probable to be matches. Most of the time, it works fairly well, but I have seen it make some decisions that were somewhat to very obviously outlandish. I have reported some of these using the feedback system.

Have you seen the postings in the forum about the API that is planned to be available once the new FamilySearch goes production live? The plan, as I have read and understood it, is for the API to support multiple desktop record managers. If we're lucky, the API will include capability to handle a client system doing the type of merging decisions you wrote about. Maybe somebody will get the chance to try out your idea. Are you a programmer?

Having done a fair amount of merging and unmerging in the current beta system, I would suggest that any automated merging should be _very_ conservative and only merge automatically on the basis of overwhelming statistical likelihood. While there is a lot of merging to do (a few dozen records per real person in some cases), it is _massively_ more difficult to unmerge than to merge. In some cases, just wrapping your head around what needs to be unmerged and where is daunting.