Friday, August 31, 2007

The Dark Side of Steve Morse's Wonderful Website

Who is Steve Morse?

Dr. Stephen P. Morse is a famous computer inventor, professor, author and researcher. He invented the great-grandfather of the Pentium chips that power today's desktop computers. In 2001 Morse brought his considerable talent, including web programming skills, to improving genealogical website searches. For this work, he has been honored with numerous awards (IAJGS 2003, IAJGS 2006, NGS 2007 and APG 2007).

What is the One Step Website?

When Morse first tried to use the Ellis Island website, he was frustrated by the inability to accomplish a powerful search in a single step. Some basic tasks took multiple, lengthy searches. Out of necessity he created a "One Step" Ellis Island search form that issued one or more searches to the Ellis Island website, combining and filtering the results. He soon began sharing his One Step Ellis Island web page with others, allowing researchers to search the Ellis Island website from his website in ways not possible on the Ellis Island website.

Soon, Morse had established an entire collection of One Step web pages that provided One Step searches of databases on EllisIsland.org, CastleGarden.org, Ancestry.com and more. He allows anyone to use these "One Step Webpages" at his website, http://www.stevemorse.org, without charge as a public service. And indeed, thousands use his website every day. Over the years hundreds of thousands have benefited from his wonderful site, including the Insider himself.

What's the Harm in That?

In July users of Morse's One Step searches of New York Naturalization records on the websites of the Italian Genealogical Group (IGG) and the German Genealogy Group (GGG) found their searches blocked and began to complain about the IGG and the GGG.

"We are not the bad guys," responded June C. DeLalio, a founding member of the IGG. "Neither IGG nor GGG blocked people from accessing our databases through Steve Morse's site." DeLalio explained that the ISP they were paying to host their websites blocked Morse. Morse's traffic overwhelmed their server. "Our ISP provider crashed which affected not only our two organizations but all of his other clients. He therefore banned Morse's access."

What is a Denial of Service Attack?

The need to block someone is not unusual for an ISP. When a website is hit with a large number of requests in a very short time frame, it can overwhelm, disable and even crash the web server. Server operators must have the ability to block a stream of requests that could bring their servers down. When bad guys try to overwhelm a server, it is called a denial-of-service (DoS) attack. It can also happen accidentally, with no malevolence intended; Wikipedia calls this an unintentional attack. "This can happen when an extremely popular website posts a prominent link to a second, less well-prepared site."

If you were an early user of PAF Insight or if you are a frequent user of FamilySearch.org, there is a good chance you have experienced being blocked. Did PAF Insight ever give you this message?

Family Search access temporarily suspended
Access has been temporarily suspended
by the Family Search server due to overloading.

Or has FamilySearch.org ever given you this message?

Your access to FamilySearch.org has been temporarily suspended. Your computer or a computer at your location has sent a large volume of requests to FamilySearch.org and as a result access to FamilySearch.org from this location has been temporarily suspended. Usually the suspension occurs when a computer uses software to automate requesting information from FamilySearch.org from any computer in your location. If this message appears again, contact the vendor who sold you your software to obtain the latest release.

The first version of PAF Insight caused an unintentional attack on FamilySearch.org by sending a large volume of requests almost simultaneously to FamilySearch.org for each name searched. FamilySearch.org now blocks traffic coming from the early versions. In later versions Ohana software cut the number of requests per search to a level that FamilySearch.org could handle.

Did One Step Mis-step?

As for One Step harming IGG/GGG, DearMYRTLE suggests that one must consider another possibility. "The ISP would have been more overwhelmed if the searches had not gone through Steve Morse's more efficient search engine. Why? Without Steve Morse's search engines, people typically spend more frustrating hours..., thereby truly bogging down ISP servers."

There are a couple of ways Morse can implement a One Step search. One is non-intrusive and one can have dramatic effects on a website. To demonstrate, let's try some Social Security Death Index (SSDI) searches using Morse's One Step SSDI.

Method 1: Build the URL

For a Build-the-URL search, Morse builds a URL and calls the target website. To see an example of the Build-the-URL method, follow these instructions.

  • Go to http://stevemorse.org/ssdi/ssdi.html
  • Set "First Name" to "starts with" and enter "S"
  • Set "Last Name" to "is exactly" and enter "Morse"
  • Set "hits/page" to "100"
  • Set "Search Engine to use:" to "RootsWeb"
  • Click Search
  • When asked "Searches involving... Do you wish to continue?", click "OK"

The results look something like the image below. The first thing to notice is that the URL is for http://ssdi.genealogy.rootsweb.com/. The important part is "rootsweb.com." The One Step webpage built a URL for RootsWeb that accomplished what you specified, and then requested that URL from RootsWeb for you. Notice that RootsWeb returned 17 results and bothered us with two advertisements, for which RootsWeb got paid.

Screen Image of Build-the-URL Search

If the link on Morse's website brings no new searchers to the target website, then a Build-the-URL search might decrease the total number of searches as DearMYRTLE suggested. But if Morse's website is really popular and really well known (which it is) and brings lots and lots of new searchers to a small, previously unknown website, then the target website can be—quite unintentionally—brought down.

Method 2: Automated Agent

For an Automated-Agent search, Morse's server stands between the user's computer and the target server, requesting multiple pages that the user doesn't see, filtering and aggregating the results before returning them to the user.

To see an example of an Automated-Agent search, change the "Search Engine to use:" to "RootsWebPlus" and repeat the same search as before.

The results look very similar, as shown in the next image. Notice, however—and this is very important—that the URL is for http://stevemorse.org/. What's going on here? It certainly looks like RootsWeb. But in fact, it is not. We sent a request to Morse's website, which made an unknown number of requests to RootsWeb, aggregated the results and returned them to us.

The situation has some technical characteristics similar to Ancestry's botched Internet Biographical Collection. (Hopefully, Morse has RootsWeb's permission, making the two situations legally different.) Advertisements are the first casualties. As we shall see, hundreds of advertisements have been removing, cheating advertisers if they pay for them and cheating RootsWeb if they don't.

Screen Image of Automated-Agent Search

Scroll through the page and you will find that instead of 17 results, there are 100 followed by a "NEXT" link. For this page alone, Morse's website sent 41 search requests to RootsWeb. (See the sidebar, "Counting the Requests," to learn how to figure out the number of requests made behind the scenes.) The 41 requests hit RootsWeb much faster than a human could send them.

Counting the Requests

Skip this sidebar if you're not interested in the nitty gritty details calculating how many search requests are made to RootsWeb for each page of the RootsWebPlus example.

Since RootsWeb doesn't allow wildcards in the first three characters, to get all given names starting with S, stevemorse.org must ask for all the name combinations from "SAA" to "SZZ". That would be 676 different search requests.

There is also the single initial S followed by a space and a middle name or initial. (Using a dash in the middle to represent a space between two letters, we can represent this as "S–A" to "S–Z"). That is another 26 combinations.

And there's the possibility of two letter first names (which we can represent as "SA–" to "SZ–"), another 26 combinations. Finally, there is the single initial "S" ("S––"). That is 729 combinations.

In practice, one can treat the space character as a letter of the alphabet coming before A. The series then becomes
"S––", "S–A", "S–B", "S–C", ..., "S–Z", (initial "S")
"SA–", "SAA", "SAB", "SAC", ..., "SAZ", (starts with SA)
"SB–", "SBA", "SBB", "SBC", ..., "SBZ", (starts with SB)
... (starts with SC through SY)
"SZ–", "SZA", "SZB", "SZC", ... "SZZ" (starts with SZ)

That makes 27 rows each with 27 combinations, which is, again, 729 total combinations.

The first page starts with "S Morse" and ends with "Samuel E Morse". That is all 27 combinations from "S––" to "S–Z" and another 14 combinations from "SA–" to "SAM". That's a total of 41 combinations, so it takes 41 searches.

The second page goes from "Samuel" to "Seymour N". That's 13 searches for the rest of the "SA" combinations, 27 for the "SB" combinations, 27 for the "SC"s, 27 for the "SD"s and 26 (all but "SEZ") from the "SE" combinations. That's 120 searches for page 2 (13+3*27+26)!!

The remaining pages can be analyzed in similar fashion. Page 3 goes from "Seymour" to "Sophie", which takes 262 searches (2+9*27+17). If you add all the searches from all the pages you'll get 728 plus the number of pages. That's more than the 729 predicted. Can you see why? If you look at the URL on each page, can you figure out how Morse knows where to start each page? On several pages look at the little table above the search results. With what you've learned, does the value for the First Name make sense now?

Click the NEXT link. Notice how much longer it takes for the second page? That's because stevemorse.org made 120 search requests to RootsWeb! The third page makes 262 requests to RootsWeb. Over all pages, over 729 search requests will be made to service your one search.

Just like PAF Insight, an Automated-Agent search can send a significant number of searches at an incredibly fast rate to the website being searched and, if enough users do it, can bring down a site as large as FamilySearch.org.

Trouble With Ancestry?

Some of you may remember Morse used to provide Automated-Agent searches of various Ancestry databases. The Insider used stevemorse.org whenever he needed to bypass Ancestry's wildcarding rules. Sadly, this feature went away. We can only assume Ancestry asked him to eliminate this capability because:

  • It was a neat feature that we don't think Morse would have taken away unless it was requested.
  • The feature required the user to break the Ancestry Terms and Conditions by disclosing their username and password.
  • The feature's instructions disclosed information that allowed inappropriate misappropriation of Ancestry's cookies.
  • A wildcard search such as "S*" would have hit Ancestry's servers with 729 near-simultaneous searches. (We don't remember for sure, but we don't think you could do a 3-character wildcard "???" that would issue 19,683 searches.)
  • Advertisements would have been suppressed. Either the ads would be "served" and the advertisers would be cheated because no one viewed ads they paid for, or the ads would not be served and Ancestry would have been cheated of the ad revenue. (Wouldn't it be ironic if Ancestry complained to Morse over this issue and then turned around and did it to others with the Internet Biographical Collection? No one has shorter memories than corporations.)
  • Having pages coming from the stevemorse.org domain that looked and felt like Ancestry pages would have been a copyright violation.
  • On the other hand, serving up results without acknowledging the owner, is probably misappropriation of intellectual property. (This and the last item are a catch22. You're damned if you do and damned if you don't.)

Conclusion

Steve Morse has been enhancing and optimizing his website for a number of years. He's had to adapt and comply with the needs of the websites for which he provides enhanced searches. He's proven himself ethical and accommodative. While it is entirely possible—even highly probable—that his website had a negative effect on the Italian Genealogical Group and the German Genealogy Group websites, we have every confidence that Morse will surely cooperate with the website owners and quite possibly collaborate with them in ways only he can.

Thursday, August 30, 2007

Ancestry.com Launches DNA Website

DNA Ancestry Advertisement

This morning a user reported on the Genealogy-DNA mailing list that Ancestry.com has launched its DNA website at http://dna.ancestry.com/.

The Insider has confirmed that the advertisement shown to the right is running on Ancestry's website, effectively announcing the new website. The move has been expected since June when Ancestry announced it had acquired Relative Genetics. The website, named DNA Ancestry, is flagged as being in "beta" stage in the header. The Insider is preparing a review for a future day.

Wednesday, August 29, 2007

Internet Biographical Collection Removed from Ancestry

Ancestry just posted this message on its 24/7 Family History Circle blog:
Earlier this week we launched the Internet Biographical Collection on Ancestry.com. Our goal was to offer members a search engine that focused primarily on genealogy resources. We intended this collection to help surface family history information that many people would not be able to find easily because it is often scattered among numerous websites across the Internet. We cached individual Web pages in an effort to preserve history – if a Web page featuring important family history information were taken down in the future, a cached version would still be available.
Many people have expressed concerns about the collection and the search engine we created on Ancestry.com. We recognize the significant time and resources members of the genealogical community invest to make their family history research available online.
Over the past few days we have reevaluated this collection’s goals, caching and crawling ability, and user experience. We have decided to remove this collection and search engine from Ancestry.com for the time being. We are still dedicated to providing family historians the online tools and aggregated records that make it easier to trace their family tree and will work to develop a solution that meets those needs in a way that will be most beneficial to our customers and the community.

Rush Indexing Project

FamilySearch Indexing just did a rush job indexing West Virginia marriages for the West Virginia Division of Culture and History (WVDCH). "Their goal is to identify individuals by county, name, and year of marriage and then link the results directly to the image of the original record where all the other data will be available for researchers," according to the FamilySearchIndexing site. To complete the project quickly, FamilySearch took a number of actions.
  • The number of fields to index is minimal.
  • Highlights weren't developed for the various register types.
  • Sample images weren't setup.
  • They sent out a message to indexers on Wednesday describing the project as "highest priority" and asking that it be done with the "utmost urgency."
  • The Download Indexing Batch button downloaded batches from this project.
  • FamilySearch sent out a message yesterday announcing that instead of seven days, indexers had one day to complete a batch and if a batch sat inactively on your computer, they would take it back and reassign it.

A representative of FamilySearch reported today that this indexing project is very nearly complete. If you attempt to download a batch for the project, don't be surprised if no work is available. This project indexed 17 digital films:

4130632 - Cabell county marriages
4130633 - Cabell county marriages
4130642 - Logan county marriages
4130680 - Braxton county marriages
4130710 - Wood county marriages
4130749 - Wood county marriages
4130803 - Lewis county marriages
4130804 - Lewis county marriages
4130811 - Jefferson county marriages
4130812 - Jefferson county marriages
4130813 - Jefferson county marriages
4130817 - Jefferson county marriages
4130818 - Jefferson county marriages
4130824 - Marshall county marriages
4130896 - Mingo county marriage bonds
4130906 - Wood county marriage consents
4130907 - Wood county marriage consents

This project is just one piece of the Vital Research Records Project of the West Virginia State Archives to place West Virginia birth, marriage and death records online. The first phase was completed in 2005 with county records for Calhoun, Gilmer, Hardy, Harrison, Mineral and Pendleton counties and statewide death certificates from 1917-54. Since then, records have been added incrementally.

"Early indications from genealogists who have begun using this new online resource are extremely positive," said Troy O. Body, WVDCH commissioner. "I encourage anyone with roots in West Virginia to take time to check out our website and make a connection with their past."

FamilySearch (aka Genealogical Society of Utah) microfilmed, scanned and indexed the records for the WVDCH. The West Virginia vital records database is available at www.wvculture.org/vrr and the FamilySearch Indexing website is at www.familysearchindexing.org.

Ancestry Trees outdo FamilySearch Trees

Back in March, FamilySearch announced that the Pedigree Resource File (PRF) database had reached over 150 million searchable names, growing at a rate of about 19 million names a year since its launch in 1999. The PRF is freely searchable online at www.familysearch.org, and notes and sources are available via low-cost CD or DVDs.

Today, Ancestry.com announced that its Ancestry Member Trees database has exceeded 275 million searchable names in its first year. Users of this free service have also attached 30 million family history records to their 3 million trees and uploaded 2 million photographs. Trees can be made public or kept private from other Ancestry subscribers. Tree contributors can give family members free access to their trees without the need for an Ancestry subscription and can specify who can change information and who can contribute photographs, stories and audio clips.

The largest Ancestry Member Tree has over a quarter-million names. Another tree has almost 3,500 photos. One user has shared his tree with more than 180 people.

Louise Roskell, from North Yorkshire, Great Britain, invited 15 family members to view her Ancestry.com tree. "[They] recalled stories about different members of the family," said Roskell. "Before I knew it, I had a wealth of photographs and was in touch with several relatives I had never met before."

Ancestry recently launched several international sites, including one in Italy, allowing a broader and broader audience of relatives to collaborate. Just yesterday Megan Smolenyak, Chief Family Historian at Ancestry, received an e-mail in Italian from a relative in Italy who found a common Italian ancestor in Smolenyak's Ancestry Member Tree.

Start building your free tree at www.ancestry.com. Click here to see the full text of the Ancestry announcement.

Tuesday, August 28, 2007

Who Is the Ancestry Insider?

The Ancestry Insider blog is the unofficial, unauthorized view of the Internet's two biggest genealogy and family history websites: Ancestry.com and FamilySearch.org. The editorial policy of the Ancestry Insider is to be both their friend and foe. He supports the attempts of both to serve the genealogical community and desires both to succeed. But both do lots of stupid things and the Insider hopes to be both their most constructive vocal critic and their most vocal constructive critic.

If you have an insider scoop that you can legally share, let the Inisder know at AncestryInsider@gmail.com.

Monday, August 27, 2007

FamilySearch Labs Updated

Two updates have occurred in as many days on the FamilySearch Labs website.

Record Search

"In case you have not seen it there is new information posted in the past week on Record Search," says product manager, Bill Mangum. The new material includes Massachusetts and New York for the 1900 Census and updates to the Ontario Death Records.

Mangum says, "keep watching...for additional data updates and for additional changes in the system."

Life Browser

"Sorry it has taken us so long to publish this update to the Life Browser prototype," says Dan Lawyer. "We'll be more diligent at publishing the updates in the future."

We’ve done a pretty major overhaul to the Life Browser. There are still many features that are ‘stubbed out’ but hopefully you can get a feel for where we are going. Here’s what’s new:
  • You can create your own life pages! Be aware however that they won’t be saved after your session. This means that if you leave the site and come back any work you did before will be gone. We hope to be able to store life pages in the future. We’ve also considered a version that let’s you save them on your own computer.
  • Artifact Palette. We’ve reworked the page and placed a palette of artifacts on the left hand side of the page.
  • Add Artifacts. From the Artifact Palette, you can add more artifacts. Give it a try.
  • Artifact Viewer. The artifact viewer lives on the right hand side of the screen. The idea is that anytime you want to view an artifact (no matter what context you’re in) the viewer slides out from the right and slides back to the right when you’re finished.
  • Evidence. Once an artifact has been added, it can also be used as supporting evidence for genealogical conclusions about a person. Just drag and click the artifact from the artifact palette and drop it on the conclusion it supports.
  • Placeholders. We’ve put in placeholders for timelines, maps, and discussion boards about a person. These are currently just images but hopefully will give you a feel for our general direction.
These are just some of the additions to the life browser. There is still a lot we’d like to do. We also have a lot of questions about the additions to this version so PLEASE use the feedback link to tell us what you think.
Visit http://labs.familysearch.org to see these and other projects. For the latest information about FamilySearch Labs projects, visit the FamilySearch Labs blog.

Top U.S. Family History Centers

"Here. Publish this Internet report," crowed the Ancestry Insider as he tossed a piece of paper on a staff member's desk.

"Where did you get this," asked the staff member. "From the FamilySearch Department?"

The Insider laughed under his breath as he shut his office door.

"Are these users? Hits? Page views? Years? Months? Weeks?"

We always know when we're not going to get an answer from the Insider. He shuts his blinds. So, for what it's worth, here's the U.S. Family History Centers with the top Internet usage, in... ah,... items-ish per..., eh, ...month.

Mesa, AZ 7,000,000
Los Angeles, CA 4,000,000
Sacramento, CA 3,000,000
Logan, Cache, UT 2,200,000
Oakland, CA 2,100,000
Idaho Falls, ID 1,800,000
Las Vegas, NV 1,300,000
Pasadena, CA 1,200,000
San Diego, CA 1,100,000
Orange, CA 1,100,000
Pocatello, ID 630,000
Ogden, UT 280,000
St George, UT 260,000
Port Angeles, WA 220,000

If you can't see the map above, try clicking here.

Friday, August 24, 2007

BYU Conference: The Digital Pipeline - Chapter 4

Stages of the Digital PipelineThis is the last installment of our report on Bill Mangum's presentation, "The Opening of the Digital Pipeline," from the final day of the BYU Genealogy and Family History Conference. In the first part, we presented the analogy of a pipeline and introduced the stages in the digital pipeline. In the second part, we talked about handling current gaps in the pipeline and the processes in the pipeline up through the scanning step. In the third part we moved down the pipeline, talking about the processes through the Infobahn stage. This week we'll finish the presentation by talking about Record Search and FamilySearch Labs.

Howard W. Hunter

“In recent years we have begun using information technology to hasten the sacred work of providing ordinances for the deceased. The role of technology in this work has been accelerated by the Lord himself, who has had a guiding hand in its development and will continue to do so. However, we stand only on the threshold of what we can do with these tools. I feel that our most enthusiastic projections can capture only a tiny glimpse of how these tools can help us—and of the eternal consequences of these efforts.”

(President Howard W. Hunter, “We Have a Work to Do,” Family History Fireside, 13 Nov. 1994)

Robot from Lost in Space,
"Warning! Warning! Will Robinson!"
The Lost in Space Robot

FamilySearch Labs

When we reached Record Search, the last stage of the pipeline, Bill Mangum displayed the Lost in Space Robot and warned us that we were now entering territory that was changing constantly and rapidly. What he was going to show us was not a released product. We would have to go to the FamilySearch development labs to see it. He also showed he was wearing slippers, a symbol that Record Search would be accessible in our homes. Mangum admitted he'd also worn a bath robe in previous presentations.

If somehow you have not heard about and visited FamilySearch Labs, then run, don't walk, to http://labs.familysearch.org. The website lists a couple of current projects:

  • Record Search - Here you can search the indexes that are produced by FamilySearch Indexing. You can also browse records that haven't yet been indexed. We'll demo this for you in just a moment.
  • Pedigree Viewer - Admit it. You've never quite given up your love for your 12-generation wall chart. Pedigrees on that teeny-tiny computer display have just never measured up—not just in size—but in utility. Locating a person on that trusty wall chart was immediate. Locating a person in PAF takes twenty-times as long. (Click here. Then here. Once more. Oops, wrong line. Start over. No, maybe try search. Wait! I remember she is RIN number 11327. Got her!) Pedigree viewer brings the large pedigree chart back. Somehow it makes it fit on the computer screen. You can see examples on the website, or you can upload a GEDCOM of your own.

The FamilySearch Labs site also hosts conceptual projects. These are still in the refinement stage and may or may not ever be released for public use.

  • Life Browser - This project is experimenting with ways to present the breadth of records and media you have about an individual. You can collect in one place the stories, pictures and records you have about a person. It doesn't take much before an ancestor becomes alive before your eyes.

The website also lists projects that have been released.

  • Venerable old FamilySearch Indexing was developed at FamilySearch Labs. Hopefully, you're quite familiar with it. I guess they list it to prove that they are more than a think tank. Hey; you may not think FamilySearch Indexing is very old, but in Internet years, its older than your dog.

Record Search

Mangum kept his own project for the very last. He is the product manager for Record Search. The product manager investigates the user needs and designs a product to meet those needs. In Record Search, all the records collected by FamilySearch (also known as the Genealogical Society of Utah) empty from the pipeline into your home or anywhere else you can access the Internet.

From the FamilySearch Labs home page, click on Record Search under Current Projects. Or go directly to http://search.labs.familysearch.org.

Currently, Record Search has not entered a formal beta release. But Mangum is allowing users to register and test it prior to the official beta, so long as the number of users doesn't exceed the capabilities of the current computer hardware. Follow these steps to register.

  1. Near the right side of the page, about half way down click on "Register to Use Record Search".
  2. Enter your E-mail address—twice—at the top of the registration window.
  3. In the spaces provided, enter your first and last names and, optionally, your state and country.
  4. To prevent automated robots from registering, a security feature requires that you view some letters and numbers and type them in the box. For the time being, automated robots aren't able to read these symbols, even though you can. Well, hopefully you can. If you can't, refresh the window. (If you use IE, press F5.)
  5. Finally, you must click the box indicating your acceptance of the conditions of use. In the same spirit of not signing any agreement you haven't read, you should first click on the words "Conditions of Use" and read the agreement. Only after you check the box will the Send Button work.
  6. Well, what are you waiting for? Click the Send button!

Your registration request is sent directly to Mangum's desk. He said he'd try and create accounts within a day of receiving the registration, but if he's out of the office, it could take several days.

Mangum then gave a demonstration of Record Search. I think I'll hold that for another day.

Gordon B. Hinckley

“Our motives are to help members of the Church and others find their roots. The doctrine of the eternal nature of the family is one of the most important and sacred teaching for us. As I learn more about my own ancestors, who worked so hard and sacrificed so much, it increases my sense of identity and deepens my commitment to honor their memory. Perhaps there has never been a time when a sense of family, of identity and self-worth, has been more important to our world.”

(President Gordon B. Hinckley, unpublished remarks, FamilySearch Internet Genealogy Service Launch, 24 May 1999)

Thursday, August 23, 2007

Dear FamilySearch, About Your Branding...

An Open Letter to Steve Anderson, FamilySearch Marketing

Dear Steve,

You joked at a recent presentation that we in the audience probably didn't think FamilySearch needed any marketing. We all laughed, we who live in the mountain west and use some aspect of FamilySearch every day.

Missionaries in Essex, VermontBut when I read this morning's Burlington Free Press (Vermont, for those of you asking), I got a chuckle. In an article titled "Missionaries work streets of Essex," writer Matt Ryan mentions the local family history center.

"The church's Family History Center...provides Internet access for members to find their ancestors on databases such as ancestry.com."

I had to laugh. O.K. I concede your point. Your Church's genealogy efforts could benefit from a unified, visible brand. I know you want to leverage the brand loyalty, trust, and respect of your splintered brands: FamilySearch.org, Personal Ancestry File (PAF), the Genealogy Society of Utah (GSU), the Salt Lake City Family History Library (FHL), and the Family History Department and the Family History Centers (FHCs) of that church with the really long name. (Let's see... Did I miss any?) Separately, the brands aren't large enough to break through the public's consciousness. Together, you'd have a real chance that any person touching their genealogy would know all about you.

FamilySearch LogoI've seen your efforts to unite under the FamilySearch brand and I commiserate with your frustrations. You've done a good job with your software and web offerings. But you're in a large bureaucracy full of people who are really attached to their current names. If I were King, here's what I would do:

  • Stick to using FamilySearch as an adjective. Besides the solid legal reasons, brands work better as adjectives than as nouns.
  • As GSU is the legal entity under which much of your work is done, rename it to The FamilySearch Genealogical Society of Utah and establish a DBA of The FamilySearch Society. Use the full name inside any markets where GSU is your known brand and the DBA everywhere else.
  • When writing about the FHL and FHCs use the terms FamilySearch Family History Library and FamilySearch Family History Centers the first time each is referred to. Afterwards, use the shortened terms FamilySearch Library and FamilySearch Centers. After three years, switch entirely to the short forms.
  • Change the name of the Family History division of the Family and Church History Department to the FamilySearch division (or FamilySearch Department).

Do these things and you'd have the FamilySearch brand in people's faces, soaking up and unifying the value of your current brands. They are worth too much to abandon.

There you have it: the opinion of an outsider. Oops. I mean I'm an outsider-insider... No, an inside-out sider... Never mind.

Good luck,
— The Ancestry Insider

Tuesday, August 21, 2007

New FamilySearch coming to Spokane

Spokane Washington Temple
Picture credit: FamilyForever.com
Dear Ancestry Insider
I live in the Spokane district and can say that we have a very active Temple and that we have been beta testers for some other materials a few years ago. We are number 11 on the list for New FamilySearch, but we don't yet know when that will be.
DMV
Dear DMV,
Thanks! I'll add this info to my table. My guess is they'll get to #11 around the first of the year. If I'm right, you should hear something during Q4 this year.
— The Insider

Sunday, August 19, 2007

BYU Conference: The Digital Pipeline - Chapter 3

Stages of the Digital PipelineThis is the third installment of our report on Bill Mangum's presentation, "The Opening of the Digital Pipeline," from the final day of the BYU Genealogy and Family History Conference. In the first part, we presented the analogy of a pipeline and introduced the stages in the digital pipeline. In the second part, we talked about handling current gaps in the pipeline and the processes in the pipeline up through the scanning step. This week we move down the pipeline through the Infobahn stage. Next week we'll finish the presentation by talking about FamilySearch Labs and Record Search.

Scanning Stats

The Granite Mountain Record Vault contains about 2.5 million rolls of microfilm estimated to contain about 20 billion names. Five years ago when scanning began, it would have taken over 120 years to convert all the records in the vault. With advancements in scanning technology, they've increased the output by over four times utilizing four scanners. An employee in the audience informed us that they were now up to eight scanners and that the scanner manufacturer is about to double the speed of the scanners.

I suppose one can take the original 120 year estimate and shorten it to 30 years with the 4-time improvement in the last five years. But I don't suppose doubling the number of scanners and doubling the scanning rate translates completely into another 4-fold improvement. Pulling a number out of the air, scanning might be complete in ten years without any further scanners. But remember that the plan is to work up to 15 scanners. Depending on how soon the scanners are added, its conceivable that scanning could be finished in 5 to 10 years.

Describe - Waypoints

The describe team creates guidance information for users of a record collection, describing why the records were created, how to use the records and what information is included. The catalog descriptions are prepared for the Family History Library Catalog.

To make record collections useful before the records are indexed, the waypoints team creates image groups. These are called waypoints as we shall see later in Record Search. Waypoints might be set up by location, date, name or other grouping, depending on how the original records are organized. Waypoints are subdivided to make the number of images in each image group small enough so users can browse through the images to find the desired records.

Indexing

While browsing digital images has some value, the records are much, much more accessible when one can type in a name and search for desired records. To make this possible, the handwriting on the digital images must be converted to text. Because technology is not capable of performing this conversion, people must view the images and manually type (or transcribe) the information. A searchable list of text is called an index, so FamilySearch calls the transcription process "indexing." www.FamilySearchIndexing.org

FamilySearch Indexing is an Internet-based system that allows volunteers to transcribe information from digital images which is then used to make searchable indexes of those images. Because FamilySearch Indexing uses the Internet to deliver images to volunteers and return transcribed text back to FamilySearch, volunteers can work anywhere they can connect to the Internet.

FamilySearch Indexing
Click to view larger format
Photograph by Robert Casey
© 2007 IRI

Once a batch of images is downloaded to your computer, the FamilySearch Indexing application allows you to work without being connected to the Internet. One batch takes about 30 to 60 minutes to complete. Text is typed into a table and highlights show the part of the image to type.

Mangum reported that there were 55,000 volunteer indexers and the number was growing fast. Because of an article in the Ensign magazine, 7,000 people signed up the previous week, temporarily overloading the servers. Over 16,000 batches were completed in a day which amounted to over 782,000 names per day. There were 67 million names done in the past year. FamilySearch expected to hit 125,000 volunteers soon. They are on the verge of producing multiple language versions of FamilySearch Indexing which will allow them to scale up to millions of names every day. However, everyone's help is necessary to make the program successful. To sign up, visit www.familysearchindexing.org

Final Assembly and Infobahn

In final assembly, miracles happen. All of the data and images come together. A large amount of behind-the-scenes operations must be performed on the data and the indexes to make them easy to search These are called standardization, normalization, and addition of derived information.

Once assembled, the Infobahn team imports the information to a test environment called staging where the team validates that it all works. It is then published to the production website, that is, the actual website.

Saturday, August 18, 2007

FGS wrap up: Ancestry News

Ancestry Shows New Ancestry Press Books at FGS

At FGS this past week, representatives from Ancestry showed samples of professionally printed books produced by its Ancestry Press self-publishing software. "We have had a lot of interest and many requests for allowing Ancestry Press users to order professionally printed and bound books," said Ancestry product manager, Kelvin Hulet. During the beta, users have had to print using desktop printers. Hulet announced that "we expect to launch this feature within a couple of weeks."
Ancestry Press Pedigree PageAncestry Press Pedigree Page
Ancestry Press allows a user to automatically generate a book from an Ancestry Member Tree. The user can then add to and modify the book before printing it. Ancestry Member Trees and Ancestry Press are two of the best values on Ancestry.com. Both are free. Pricing for printed and bound books have not been announced.
Ancestry Press Family Group Record PageAncestry Press Family Group Record Page
The software can automatically generate a basic book containing a pedigree chart, family group records for the families on the pedigree, and time lines for the individuals. Tools allow addition of photographs and other customizations.
Ancestry Press Individual Timeline PageAncestry Press Individual Timeline Page customized with additional photographs
Sources inside Ancestry say employees have been encouraged to test the process, generating test books from their own family trees. The sample book shown at FGS was the result of one volunteer's efforts.
Ancestry Press Census Record PageAncestry Press Census Record Page. Subscription owners can attach records to people in their tree and Ancestry Press automatically incorporates those records.
Ancestry Press Page with a digitized mapAncestry Press Page with a digitized map
(Some information for this article was taken from the Ancestry Press blog.)

Ancestry and NEHGS Collaboration

There is still no information on either organization's website about the announced collaboration between Ancestry.com and The New England Historic Genealogical Society (NEHGS). The first part of the press release reads,
Boston, MA & Provo, Aug 16, 2007 - The New England Historic Genealogical Society (NEHGS) and Ancestry.com today announce a new collaboration that will bring together and make available for the first time the enormous resources of both institutions in an effort to continue fostering a growing national interest in family history and genealogy.
As part of the relationship, NEHGS, the country’s largest and oldest non-profit society and Ancestry.com, the largest online family history website, will offer joint access to some of most important family history information available anywhere. While details of the collaboration will be outlined in the weeks to come, it is planned to include special membership opportunities that combine Ancestry.com’s repository of five billion names and 24,000 databases and titles and some of NEHGS’ most significant genealogical publications and services.
The complete text of the announcement can be read on the GenealogyBlog.

FGS wrap up: Triplets of different mothers

In one of the most amazing stories to arise out of the FGS Conference, three prominent figures in the genealogy community learned that they are identical triplets. Amazingly, each of the three has separate biological parents and were born at separate locations and on different dates.
Leland Meitzler, Halvor Moorshead and David Liffereth Leland Meitzler, Halvor Moorshead and David Liffereth (Credit: Leland Meitzler)
"Medical mathematicians are at a loss to explain this," says Leland Meitzer, cofounder and writer at Heritage Quest Magazine and the GenealogyBlog.com.
Fellow triplet, Family Chronicle editor and publisher, Halvor Moorshead, added, "the mathematical probabilities of this happening for twins are astronomic. And for this to happen with triplets. It boggles the mind."
"After the Fogelberg, Weisberg case a number of years ago, I thought I would never see another in my lifetime," commented David Liffereth, president of World Vital Records. "And now to have it happen to me. And with two guys I admire so much. I can't tell you what this means to me."
Genealogy Management software companies are scrambling to patch their programs to handle this unique situation, but spokespersons from several told the Insider that they would have fixes shortly. "Fortunately, Next Bio Gen 2008 already handles this case," said a spokesperson. "The resulting hourglass chart is, quite frankly, a work of art, a thing of incredible beauty."
"Credit goes to the DNA vendors at the conference. This is the discovery of a lifetime. No matter what else happens from this point on, we will always have each other. And we will always have FGS 2007."
[Warning: We're tired of handling all the email from readers when the Ancestry Insider makes an absurdly impossible claim. He thinks he's funny. (See, for example, these comments.) No, identical triplets can not have separate biological mothers. Please don't write us. We think the Insider made up most--no--all of this. The quotes. The product names. The people. The photograph. We're not even sure he boarded his connecting flight from Vegas to Ft. Wayne. If you write the Insider and he makes fun of you, don't say we didn't warn you.
--The Ancestry Insider Staff]

Building Successfully Imploded Near FHL

The destruction of the Key West Bank Tower one block from the Family History Library occurred this morning as planned. Click on a camera angle, below, to see a replay of the event.

Video courtesey ksl.com. Trouble viewing? Try their website.

Friday, August 17, 2007

New FamilySearch coming to Billings, Montana

Billings Montana Temple A spokesperson for FamilySearch has confirmed the report on FHCNET that new FamilySearch is coming to the Billings Montana Temple. Our post showing locations with New FamilySearch will be updated shortly. The full text of the FHCNET post follows.
From: Winter Owl
Date: Wed Aug 15, 2007 3:20 pm
Title: New rollout
Beginning Tuesday, August 28, the new process for printing temple name cards using Family Ordinance Requests will be used in the Billings Montana Temple. New FamilySearch will be available to the general membership of the Church who live in the Billings Montana temple district early in the morning on the previous Saturday, August 25, 2007. No additional notice will be sent to priesthood leaders or members of the Church in the Billings Montana temple district concerning these dates.
We ask that family history consultants and center directors encourage members who have existing TempleReady disks to take them to the temple and have their temple name cards printed before August 28. Beginning immediately, please do not create any new TempleReady disks in your family history centers for processing at the Billings Montana Temple.
If you have not done so already, please complete the new FamilySearch online training prior to August 28 in preparation for helping members to use new FamilySearch as part of the new process for preparing ancestral names for the temple. Please contact FamilySearch Support by e-mail or phone if you have any questions or problems.

Digitizing Allen County Library family history books

The Allen County Public Library (ACPL), co-sponsor of this year's FGS conference in Fort Wayne, Indiana this week, announced an agreement with FamilySearch and the Family History Library (FHL) to allow digitization of books from its local and family history collection for publication on the web. The books will be published as part of the Family History Archives (FHA) collection on the website of Brigham Young University's (BYU) Harold B. Lee Library (HBLL).

The FHA project seeks to digitize local and family history books worldwide. Started at BYU over two years ago, the project is now headquartered in FamilySearch offices in the Orem, Utah Canyon Park Technology Center. The FHA project utilizes volunteers and is digitizing books with expired copyrights available at the HBLL, the FHL, and now, the ACPL.

"Publishing those collections from the three libraries involved will make a significant and attractive family history digital library online for genealogists and historians," said David Rencher, director of Records and Information for FamilySearch. Among these institutions are over 100,000 local and family histories.

The FHL and the ACPL are the two largest genealogical libraries in North America. "I believe the strength of our two well known, well used, and well loved institutions working together is a terrific benefit to the genealogical community," said ACPL's Genealogy Center Manager, Curt Witcher.

The Family History Archive is every-word searchable and currently contains over 5,000 titles. Access the collection at www.familyhistoryarchive.byu.edu. See the complete announcement on familysearch.org.

Records Access Genesis

The Ancestry Insider is a slave driver. He wants his latest BYU Conference report published today. He wants all the news from the FGS conference announced. He wants the other hot news out. He wants his recorded commentaries on various topics transcribed and released. That's all easy for him to say, lounging around in Ft. Wayne.

Records Access Genesis

As we previously reported, FamilySearch began distributing a Request For Information (RFI) at FGS on Wednesday. Always trying to look superior to others, the Insider wanted us to make certain we mentioned that their RFI is actually an RFP, a Request For Proposals. (He's a perfectionist. But he's a shallow, superficial perfectionist.) Called the Records Access Genesis Project ("RAG," as the Insider is calling it), FamilySearch wishes to accelerate online record access through cooperative agreements with record owners and Internet publishers.

FamilySearch is proposing a business model for Internet publishers that Ancestry.com has purportedly already evaluated and rejected. The model proposes that FamilySearch provide digitized images to publishers, who would index the records and give FamilySearch their own copy of the index. Both would then publish the index on the Internet. Users on the FamilySearch website would be divided into two groups.

Two Groups

The first group consists of the Church of Jesus Christ of Latter-day Saints, its members, its employees, patrons at its Salt Lake Family History Library, its branch Family History Centers, Genealogical Society of Utah employees and volunteers, users at BYU and other Church owned schools, and probably other users inside the Church's firewall, such as its Deseret Industries goodwill stores. This group gets free access to the images as well as the index.

The second group, everyone else, gets free access to the index, but if they wish to see the original images, links on the FamilySearch website lead to the partnering Internet publisher's website. The partner can then monetize access to the images.

Internet Publishers

For Internet publishers, the problem with the model is the cost of indexing. As FamilySearch itself is finding, creating digital images is cheap, creating indexes is very expensive. A secondary issue is the amount of data FamilySearch likes to transcribe. Past and current FamilySearch indexing projects have transcribed most useful data from records, making access to images less necessary. On the other hand, Internet publisher Ancestry.com often minimizes indexing expenses by transcribing only enough data to allow easy finding of records, forcing users to view images to access the remaining genealogically useful data.

Still, FamilySearch feels partnerships provide value to RAG partners over and above commercializing the images. While Ancestry.com has only recently started expanding internationally, FamilySearch brings established relationships with record owners worldwide. Partners also enjoy increased exposure through the links from FamilySearch's well-trafficked website. Touted benefits that are not immediately understandable are "preserving or providing access to data" and "leverage an open platform." In the context of David Rencher's presentation, these statements could mean that FamilySearch is even willing to host the indexes and images for a partner while still allowing the partner to control and monetize image access.

While RAG partnerships with large established Internet publishers such as Ancestry.com may not make sense, partnerships might make sense for genealogical societies, volunteer-rich/cash-poor organizations, new start-ups, and publishers in markets not yet served by established publishers.

We're out of time and we haven't yet examined the business case for record owners. (We think it makes sense.)

To see the RFP for yourself, click for the cover letter, and click for the proposal (75 pages).

Thursday, August 16, 2007

new FamilySearch in Billings; FGS Reports

Just a quick report. There is an unconfirmed report that new FamilySearch is being rolled out to the Billings Temple District before the end of the month. We'll try to confirm this.

At FGS David Rencher gave a keynote titled "Societies Going Virtual." You can see the slides at http://www.familysearch.org/eng/Library/Education/societies_going_virtual_David_E_Rencher.pdf. While these slides are from FGS two years ago, the Insider says they're mostly unchanged.

Also at FGS, FamilySearch made an announcement this morning concerning the Records Access program. Dick Eastman got an advance copy of the announcement. (The Insider called in to see if his advance copy had arrived. We lied and said the mail hadn't arrived yet.) For now, read Eastman's announcement.

We've also learned that Ancestry.com is sending all its employees packing their trunks tonight. Yep, they've rented a local water park for the evening.

Half a Negro Boy

Dear readers,

Every so often you see something that eloquently expresses why you do genealogy. This short video (below) is such a piece for me. Titled "Half a Negro Boy named Coleman," it tells the story of Megan Smolenyak, Chief Family Historian at Ancestry.com, and her research into the ancestry of Al Sharpton and the terrible, wonderful, incredible experience it turned into.


If you can't see the video above, try clicking here.

Coleman, sold in halves as property! It makes me sick to my stomach. No wonder Smolenyak says that "Coleman is just screaming for his story to be told."

While lacking the pathos of slavery, I've felt the same sort of feelings as I have learned of my own ancestors. They want to be remembered. They deserve to be remembered.

If you've done genealogy long enough, you've felt them reaching towards you. Call it guidance. Call it luck. Call it inspiration. Call it serendipity. Call it communication. Call it supernatural. Al Sharpton called it destiny. "As I walked around where my great grandfather and probably his parents actually lived and worked, it gives you a real sense of destiny, that you had to find this. It's in your veins."

And as you reach back towards them, your life is enriched in unexpected ways.

"When you understand what your ancestors have endured and what they've overcome," says Smolenyak, "no matter what you're going through in your own life, I think it gives you strength. You realize…we all come from the strong stock. And when you learn about your ancestors and you learn what they really experienced and you bring them back to life in a sense as people, you realize that their fortitude is flowing in your veins and you've got the same capability. It's a very positive thing."

The effect on Sharpton and his fight for African-Americans was electric. "It has redoubled my commitment. I get up every day knowing that in my actual veins flows the blood of someone who probably wished the day would come that someone would do something about it."

We can all take lessons from our ancestry as Sharpton did from his. He noted, "we have an obligation to those that never had the options we have but held on so that we could exist. [Coleman] is my grandfather's father. It gives you a boundary in life. Because before you get too into yourself or too out of line, you start thinking, what would they think that we've done with the struggle. … Think about it every morning. It makes you operate with a certain discipline all day."

Reverend, I couldn't have said it better.

-- The Insider

Wednesday, August 15, 2007

Sneak peek at *User's Guide to New FamilySearch*

The Ancestry Insider received this tip from a reader.

To: The Ancestry Insider
From: AJ
Date: 13 Aug 2007
Subject: Take a look at this, sneak peek into the forthcoming 'User's Guide to New FamilySearch'

Not sure if that title will hold, but whatever it was, it is in this thread on the LDS Tech forum, the first post has the quotes from the new book. Everyone will get it with the rollout as it arrives in their area, apparently.

http://beta.tech.lds.org/forum/showthread.php?p=5380#post5380

Dear AJ,

Call me picky. My copy of the manual has 2 more words in the title than yours ("A" and "the"). Take a look for yourself.Image of the User's Guide Here's something to try when you have nothing better to do. Order 36798 from distribution and see if you get anything.

-- The Insider

Building Explosion to Close Family History Library

This Saturday, 18-Aug-2007, the Family History Library (FHL) in Salt Lake City, Utah will be closed until noon according to a FamilySearch spokesperson. At 6:30 AM that morning the Key Bank Tower, a block away from the library, will be exploded. Technically, explosions will be used to implode the building. Because of the dust cloud that explodes from the imploding building, roads within the 9-block area shown below will be closed, including the road in front of the FHL. Google map showing 9-block dust zoneAll buildings on the same block will be evacuated except the Marriott Motel. Presumably, most employees of FamilySearch will have front row seats, as they are officed in the Joseph Smith Memorial Building and the Church Office Building. However, those in buildings within the 9-block dust zone will be required to remain indoors for hours after the blast to allow for safety inspections and dust settlement.

The public is asked to avoid coming to the site to witness the event. Local media will be covering the event live, starting at 6:15 AM. TV viewers will be able to watch the event from multiple cameras, all with better vantage than eyewitnesses, and will be able to watch the multiple replays before heading back to bed.

The building destruction is to make way for City Creek Center, an urban renewal project of the Church of Jesus Christ of Latter-day Saints which also owns the FHL. Part of the project is new underground parking. Having a common owner, can we hope that the FHL will give parking validations?

Diagram of planned underground parking Diagram of planned underground parking

Tuesday, August 14, 2007

Ancestry releases FTM 2008 (updated post)

The speed of the Internet is amazing. Trying to make timely announcements is difficult and trying to publish a scoop is nearly impossible. Fortunately, the speed of the Internet also makes it possible to be informed of one's mistakes. So here is "take 2" of this post.   --The Insider Staff

FTM 2008 Logo.Ancestry.com announced today the release of Family Tree Maker (FTM) 2008. The large number of new features comes with a price. During the beta, users of previous versions of FTM complained about the huge disparity in user interaction from previous versions. New users, however, are likely to experience greater satisfaction than new users of previous versions.

Users who previously purchased FTM as an inexpensive method of obtaining an Ancestry.com subscription may also be disappointed. FTM 2008/Ancestry.com bundles will be available in October through retail outlets with Ancestry.com subscriptions for 1, 3 and 6 months with list prices of $29.95, $69.95 and $99.95, respectively. Since an annual Ancestry subscription is currently available online for $12.95 a month, it only makes sense to purchase FTM 2008 at list price if you want FTM.

Perhaps another time we'll explore more about FTM 2008, including which bundle to purchase, and what discounts are being offered by retailers.

Family Tree Maker 2008 is available now at the Ancestry Store for $39.95, which includes a 14-day subscription to Ancestry.com.

Ancestry.com maintenance during night of August 15

A source inside Ancestry.com informs me that they will perform a hardware upgrade Wednesday night, 15-Aug-2007 starting at 11 PM MDT. They expect to be complete before Thursday morning, 16-Aug-2007 at 2 AM MDT. If all goes well, during the 3 hour time frame a user might experience an error on one page followed some time later by an error on another page.

The outage will be instantaneous, so you can immediately continue. Should you experience an error, first try refreshing the page. IE users can press F5 on their keyboard. If necessary as a second step, IE users can press Ctrl-F5 (press and hold down the Ctrl key, press F5, release the Ctrl key). This "super-refresh" gets new copies of parts of the page not normally refreshed.

If refreshing the page doesn't correct the error, redo the operation. Operations might be, for example, adding a person to a tree, searching for a record, or attaching a record to a person in a tree. Clicking links to get to a place where you can redo the operation is probably preferred to clicking the Back button. If that is not possible, click Back until you reach a point where you can start the operation over again.

Contact the Ancestry support website should you experience errors that can't be corrected.

Monday, August 13, 2007

Free Genealogy Youth Fair Aug. 18 in Fort Wayne

If you and your children are in the Fort Wayne, Indiana area this Saturday, August 18th, consider this fun-sounding activity. The 2007 FGS Youth Fair is free and open to the public. Children are invited to the Allen County Main Library (900 Library Plaza, Fort Wayne, Indiana) from 10 AM to 3 PM "for a fun-filled day of performances, demonstrations, and hands-on activities."
Events and exhibits include:
  • A day in the life of a Union soldier and display of toys and clothing by re-enactors from the 44th Indiana Civil War Historical Association
  • Pioneer music performed by Settlers, Inc.
  • Bagpipe music performed by the Fort Wayne Scottish Pipes and Drums
  • Wool spinning demonstrations by the Fort Wayne Flax and Fleecers Guild
  • Short talks on Local Historic Figures by Historic Fort Wayne
  • Period costumes provided by the Fort Wayne Civic Theatre. Photos of children in costume will be taken and printed on the spot. Scrapbooking materials and assistance will also be provided so you can create a personalized memento.
  • Allen County Genealogical Society of Indiana
  • Allen County 4-H
  • Girl Scouts of Limberlost Council
  • Boy Scouts of the Anthony Wayne Area Council
  • Shipshewana Chapter of the Children of the American Revolution

The fair is sponsored by The Generations Network, parent company of Ancestry.com and other genealogy websites and products. (Source: FGS website)

Thursday, August 9, 2007

The Digital Pipeline - Chapter 2

This is the second in a series of four installments on Bill Mangum's presentation, "The Opening of the Digital Pipeline," from the final day of the BYU Genealogy and Family History Conference. Last week in the first chapter we presented the analogy of a pipeline and introduced the stages in the digital pipeline. This week we'll talk about handling gaps in the pipeline and we'll explain the first several processes in the pipeline. In the third chapter we'll continue down the pipeline to the Infobahn stage. In the fourth and final chapter, we'll finish relating Mangum's presentation, talking about Record Search and FamilySearch Labs.

Swivel processes at pipeline gaps
(Credit: chair clipart)

Swivel Chairs

The current pipeline has gaps where the processes are not yet complete. "We have what we call swivel processes," explained Bill Mangum. These are manual processes where someone must accept the output of the pipe at one of the gaps, manually perform a task, swivel around, and send the data on down the pipeline. "As we go, we'll fill in the gaps with automated tasks."

Current Pipeline Processes

Collection management attempts to find record collections at government agencies, archives and other locations that include most of the people in the area in the desired time frame.

Record Services works with archives and various other record custodians to gain permission to acquire copies of the desired records. They write the agreements and assure that all legal requirements are accounted for. They manage the relationships between FamilySearch and the record custodians.

Since 1938 FamilySearch and its predecessor, the Genealogical Society of Utah (GSU), have been microfilming records. In fact, GSU was involved with microfilm from its inception and drove many of the standards that are used.

Cameras capture records across the nation, and in fact, throughout the world. To see the extensive list of locations where FamilySearch (aka GSU) is capturing records, visit the GSU Website at http://www.gensocietyofutah.org/ and click on Worldwide Activities. GSU currently employs over 240 cameras. If I heard correctly, over 200 of the cameras are now digital.

Diagram of camera stationDigital cameras have several additional advantages. Operators can check their work immediately and re-shoot as necessary without extra time and travel. The lights used on the copy stands are cool to the touch, which is much better for original documents. Operators store the images to 1/2 terabyte drives which they send to Salt Lake. The vision is that these images will one day flow through the pipeline so quickly, they will be available to users in a very short time.

Image Processing receives the images in Salt Lake. They receive about a terabyte of images each week. They create an archival copy of the image, de-skew any tilted images, adjust brightness and contrast if necessary, and create a publication copy for the Internet.

Photo of the Granite Mountain VaultScanning is also proceeding on the existing microfilm collection at the granite mountain vault. An employee of the scanning team in the audience let Mangum know that the number of scanners was now up to 8. They are working their way up to 15. While the scanners used to scan one page at a time and required an operator on each scanner checking every page, the scanners now image an entire film as one long strip and automatically divides the pages. The audience member told us they only require 2 operators to keep all the scanners active.

Next week we'll finish up scanning and move on down the pipeline.

Wednesday, August 8, 2007

TGN Changes RootsWeb Branding

The Generations Network (TGN), parent company of Ancestry.com, Genealogy.com and RootsWeb.com recently extended its common branding initiative to RootsWeb. The graphics below show the old and the new page headers. The new header confirms a rumor the Insider heard that RootsWeb would be re-branded as a community operated by Ancestry. The disappearance of the word free from the branding was troubling, although the Insider has been assured that the RootsWeb community would continue to be available at no charge.

Old RootsWeb Page Header Old Header

New RootsWeb Page HeaderNew Header


Freepage websites also got new headers and footers. The Insider appreciates the attempt at unified branding across TGN properties, and he feels the new header is more functional than before... And he remembers the days when awful advertisements appeared at the top of pages and lame advertisements appeared at the bottom... And he can appreciate the greenness of nature, leaves, responsible corporate behavior, etc....

"It looks sick," the Insider blurted out, "unless the Freepages website has a white background. Sick, sick, sick!" The previous header, with its neutral hues, did seem to go better with the odd variety of personalized Freepages websites.

Free Pages Old HeaderFree Pages Old FooterFree Pages Old Header and Footer



Free Pages New HeaderFree Pages New Footer Free Pages New Header and Footer

Tuesday, August 7, 2007

Premium Websites Finally Available @ FHCs

FamilySearch informed Godfrey Memorial Library Director Richard Black a week ago that they had worked out the problems with access to the Godfrey website for Family History Centers (FHCs). Today, several FHCs reported online that the FamilySearch department was calling centers individually to walk them through the setup. Renee Zamora of the Alpine Utah FHC and "Myrtle" of the Bellevue Washington FHC explain the steps they took to get access. The menu screen (above) appears exactly as the Insider previously reported, with the addition of support information.

Friday, August 3, 2007

Last Day of BYU Conference

On the final day of the BYU Genealogy and Family History Conference, FamilySearch announced that conference attendees would be able to keep their New FamilySearch accounts. Attendees were given strict instructions not to submit names until given permission. Since New FamilySearch makes name submission so slick, the Insider hopes attendees don't slip.

The Digital Pipeline - Chapter 1

This is the first in a series of four installments on Bill Mangum's presentation, "The Opening of the Digital Pipeline," from the final day of the BYU Genealogy and Family History Conference. Today we'll present the analogy of a pipeline and introduce the stages in the digital pipeline. Next week, in the second chapter, we'll talk about handling gaps in the pipeline and we'll start down the pipeline, explaining the first several processes. In the third chapter we'll continue down the pipeline to the Infobahn stage. In the fourth and final chapter, we'll finish relating Mangum's presentation, talking about Record Search and FamilySearch Labs.

Bill Mangum, senior product manager for Record Search, made a presentation roughly similar to his presentations with Rich Running at previous conferences. Mangum began by showing a Dilbert cartoon:

Mangum's point was that he would be telling us about things, some of which do not yet exist. And by the end of the presentation, even the Insider was heard mumbling, "I gotta get me some of that."

Stages of the Digital Pipeline
Photo credit: BLM/TAPS

Full Function Pipeline

Mangum showed a photograph of a pipeline similar to the one to the left. While Mangum chose a photo of the trans-Alaska pipeline running across ugly desert permafrost, the Insider insisted on the more picturesque Denali Fault area.

This diagram represents a fully functional, largely automated process that quickly takes new records, processes them through all the stages of the pipeline (collection management, acquisitions, legal services, scanning, describe, waypointing, indexing, final assembly, Infobahn and record search), making the records available quickly and efficiently to a world-wide audience.

Unfortunately, today the pipeline is a pipe dream. Fortunately, progress is being planned, managed, measured and achieved.

The Current Pipeline Has GapsTools of the Pipeline

The FamilySearch development team uses a tool for Business Progress (or Performance) Management (BPM) to aid in construction of the pipeline. FamilySearch uses BPM to define what is to be done, track where it is at, know when it is done, and report on how well it went.

The Current Pipeline

Currently, there are gaps in the pipeline, visually represented in the diagram to the right. Some parts of the process are either not yet automated or not yet optimized. Next week, we'll learn how these gaps are managed and explain a bit about each stage in the pipeline.