Tuesday, July 8, 2008

Inside the problems of Ancestry's New Search

Upon hearing a rumor that Ancestry.com intended to turn off its Old Search sooner than later, I posted an article pinging you, my readers, to see if you thought New Search was ready. Comments ran 4 against and 1 for. I also posted an online poll asking, "Is Ancestry's New Search ready to replace Old Search?" Because online polls don't use random sampling, the results are unsound. But when 18 of 19 voters say that New Search isn't ready to replace Old Search, you'd think Ancestry would sit up and take notice.

And notice they did. After that post, Ancestry's PR machine went into action. Kendall Hulet, Ancestry's head of product management, did an interview for DearMYRTLE's Family History Hour 1 July 2008. Reaction from listeners looks very positive. I feel like Mary Poppins in the final scenes of her stay with the Banks family. Mr. Banks is finally connecting with his family. The wind is changing. It must be time for me to move on. What's that you say? A tear? Sigh....

Then, to my surprise Ancestry sent the Ancestry Insider an invitation for an interview with Hulet. The Ancestry Insider wrote back and gladly accepted. I had to laugh, though. After carefully doing our dance of independence through the Ancestry Insider's e-mail address, PR booked the meeting like normal through Outlook and Exchange. I will miss all the great people in our PR department. Mike Ward is not only a good friend, he's a 5th cousin! But I digress...

Monday I brought my laundry list of New Search complaints culled from my own experience as well as what you've shared, directly and elsewhere on the net. Hulet was very open and freely shared some problems and solutions of Ancestry's New Search.

New Home Page Forces New Search

When Ancestry's new home page (for logged-in users) was released 2-July-2008, complaints started popping up online about New Search! Since the home page and the search are independent, I was confused.

Hulet said that some of the links on new home page inadvertently switch users' search preference to New Search. While the user can manually switch back, using the links on the home page will again force the user to New Search. Hulet characterized the issue as of such high priority that a fix was being pushed forward as quickly as possible. Hulet expects it will happen today.

There were two sets of links that could have been incorporated into the new home page. The legacy set would respond to the users' search preference and should have been used on the new home page. The new set would override the users' search preference. Some of these links were inadvertently used.

I poked around a bit and I think these are the links of which Hulet spoke. clicking a link in the first column will preserve your search setting. Clicking a link in the second will switch you to new search.

Legacy Links New Search Links
Birth, Marriage & Death Records Birth, Marriage & Death Records
U.S. Immigration Collection U.S. Immigration Collection

 

Hulet warned that the fix has the side effect of undoing some users custom links, but deemed the fix important enough to justify this side-effect.

More False-Positives

Many people, including myself, have noticed that New Search's exact searches return a lot of false-positives, which is to say, matches that aren't matches. Performing the same search on old and new gives more results with new. That would be good, except the additional matches are all non-matches. Hulet admitted this is a problem. Old search looks for "proximity" among the search terms. In other words, the terms must appear close to one another on the page. New search does not, so it returns more and worse results. Hulet all but promised that a fix would be released for this problem some time this week so that new search also looks at proximity.

Hulet also mentioned they are close to releasing a bug fix for another issue that produces bad results. If one searches for a name using a wildcard, like "Joh* Smith," the wildcard causes the name parser to throw a gasket trying to divide the given name and the surname. The fix is in the pipeline and will likely roll live this week or next. In the interim, to perform a wildcard search on a name, use the Advanced search form which has separate first and last name fields. To switch between normal and advanced, click on Advanced at the top-right corner of the search field.

Hulet is hopeful that these corrections will fix all the instances where new search returns more false-positives than old search, but acknowledged there's no guarantee. These two bugs are so predominant, Ancestry won't be able to tell if any further, minor bugs exist until they take care of these two.

I'll have more of the issues I discussed with Kendall Hulet in an upcoming article. Stay tuned...

12 comments:

  1. Dear A.I.-

    I've been away for a bit and just now catching up on the blogs. Thanks for the update on Ancestry's "New Search." It seems there *might* be cause for some hope that Ancestry *may* address some of the many concerns about the poor performance of the New Search GUI and search engine (not to mention the sprawling, less-specific, pop-up laden, tree-promoting New Home Page. Aacckk!).

    However, I think you are reading some *very* different responses to Kendall Hulet's interview with DearMYRTLE than I. The blogs I've read continue to be strongly anti-New Search. For example, see the comments at Ancestry's own blog at:

    http://blogs.ancestry.com/ancestry/2008/07/03/follow-up-on-the-new-ancestrycom-homepage/#comments

    I strongly agree with the majority of these comments and specific critiques of New Search, and urge you to read them if you have not already. I am most particularly in agreement with the multi-part explanation and analysis of the issues from reader Jerry Bryan (see his comments nos. 14, 18, 26, 27, 28, 30).

    FYI, it appears (comment 51) that Kendall Hulet has read them also and will respond shortly.

    So, please, don't assume that it's "time for you to move on" to other issues. I can't put it better than Ancestry-blog reader Jerry Bryan did recently:

    "Not to put too fine a point on it, but I’m a much better fuzzy search engine than is ancestry.com. But for me to be my own fuzzy search engine, I need for ancestry.com to be a very good exact search engine that I can use when I’m being my own fuzzy search engine. Then, I can try spelling variations on my own and I can use wildcards more effectively. I know how my surnames are typically misspelled and mis-transcribed. I would even love to search with regular expressions (nobody talks about ancestry supporting regular expressions, but they are way better even than wild cards for doing difficult searches). I would love always to be able to reduce, not increase the number of hits.

    "In looking at the requests for improvements through the years that seem to be regularly ignored, nearly all of them have to do with making exact searches better rather than making fuzzy searches better. The three character limit on wild card searches is an example of a frequent request that has been long ignored. But ancestry’s vision is only to make fuzzy searches better, not to make exact searches better. There was not even the slightest hint or the slightest mention in Kendall’s interview about making exact searches better. So I would say to ancestry, please, please, please quit trying to make fuzzy searches better. That’s really just the wrong vision, and it won’t work anyway. Instead, please, please, please make exact searchers better. Please internalize and embrace the fact that good quality exact searches with lots of good indexes and lots of good searching options in the search interface are what serious researchers really need. It’s ok to make fuzzy searches better if you want to. But make exact searches better first. Please!"

    Thanks for your time.

    ReplyDelete
  2. I use Ancestry mostly for Census record searches. Things like oneworldtree etc. are riddled with bad data and useless so I go back to basics to get accuracy.

    I have found a number of indexing problems with census searching. The people at the National Archives in DC confirm that they get a lot of complaints about errors in the Ancestry indexing for census records.

    Some specific cases:

    It returned a roll and page for a name. The image did not have the name. I viewed the roll and image at the Archives and the name was there. Looks like what Ancestry returned was the correct page but in the wrong county.

    Searching Barren County KY, Ancestry returned one record. The paper index at the Archives however showed two records. Checking the film, indeed, there was two records and for different people. The old manual paper index was correct and Ancestry was wrong.

    ReplyDelete
  3. I am continually amazed that TGN (Ancestry.com, Family Tree Maker) keeps on releasing poorly tested, poorly thought out "enhancements" and products. The errors with the exact search and the wildcards should have been found with a first pass, high level testing of the new search interface. I suspect some manager pushed it saying that since they didn't change the search engine, then there should be no problems. Duh! My advice to the programmers and test team is to push back when early releases are suggested!! -lhm

    ReplyDelete
  4. Insider,

    This is play with 3 actors:

    1) Ancestry - known by its emphasis on marketing driven initiatitves instead of customer driven ones (proven by spending 4-1 on marketing vs. data acquisition by Mr. Sullivan's figures from a speech earlier this year), an attendant refusal to seriously engage its most knowledgeable (and demanding) customers by responding to complaints in the manner a customer wishes (like on the message boards), and preference for the 2nd type of customer described below. There would seem a good chance that the irony is that customer turnover is very high, and thus Ancestry's 4-1 spending on marketing vs. data acquisition (or fixing problems) which only feeds the fire.

    2) Type 1 customer - this is the serious, traditional type of genealogist who values evidence based assertions over family lore and legend, and research in original documents, and doesn't think one can do proper genealogy with only online searching. This type of customer wants more original sources added regularly, and the ability to make meaningful exact searches on various databases to really drill down and make sure nothing is hidden in the forest. These type of customers are not very interested in Ancestry's other products like family trees, publishing etc. They are however probably more likely to renew than the other type of customer below.

    3) Type 2 customer - this is the unknowledgeable most common type of "genealogist" who rarely if ever darkens the door of a brick and mortar repository, and whose lack of genealogical knowledge impels him/her to contact others for the information they have found, regardless of the veracity or sparseness of sources for those assertions found. Despite the fact that Ancestry has quite a bit of good learning material as well as books for sale that could help this type of customer, that is just too much effort or outside of his/her abilities. While of course all type 1 customers were type 2 at in the beginning, the truth is that only a relative handful make the progression in learning and ability. But this customer is more likely to fall for marketing "gee whiz" gimmickry, fancy new products, and search functions that produce too many results instead of too few. The problem and irony of this though is that this type of customer/"genealogist" is least able to disambiguate conflicting genealogical identities when there are too many candidates.


    This situation of the divide between these two types of customers is never going to change. But Ancestry does a real disservice to type 1s by dumbing down everything and trying to force them into using search algorithms and strategies that are not effective for experienced genealogists. The easy solution though, which Ancestry refuses to adopt, is to have two different interfaces available, at least below the surface. By this I mean do whatever they want with "new search", and even "old search", BUT GIVE #1S THE SAME ABILITY TO MAKE EXACT SEARCHES ON EACH AND EVERY SEARCH FIELD IN ANY INDIVIDUAL DATABASE!.

    This means for example, that those of us who want to use very specific searches on only one database, which is the only way to generate meaningful research logs, should be able to go the 1840 US census, and find a search form that allows for exact searching by field instead of just one global exact search ability, or having to go through the main site search form for any real exact searching. Then *we* can control the search. Which is the bottom line - *we* make the search do what *we* want individually, instead of Ancestry trying to tell us what *they* think we want, or should want.


    Since this is already a long reply I will add something else. Which is that I see a 4 part plan for Ancestry to have a much better relationship with type 1 customers in the future:

    1) give us the ability the define searches as we wish, and on individual databases;
    2) respond in the message boards to complaints there regularly, instead of very sporadically;
    3) float trial balloons for new ideas well before the implementation/beta phase (i.e. during concept phase);
    4) spend more on acquisition of original sources.


    And here is something Ancestry should keep well in mind: type 2 customers need us type 1s, and very badly. They need us to put source-based accurate trees online, and they need us to help them with their queries and problems. Otherwise all that happens is that one person puts some unsubstantiated lineage online and 1000 people copy it and perpetuate the errors.

    Mike

    ReplyDelete
  5. One key lack in the new search *results* is inability to sort by date, sort alphabetically by name returned, or by database. Thus the user has no way to filter the many thousand (in many cases) erroneous results, such as vitals for wrong country, post-1880 databases for a person who died in 1874, and so forth.

    ReplyDelete
  6. Regarding the post with the 3 actors. I am a type 1 customer BUT I decided to drop my 10+ year subscription. I monitor to see what is going on to determine if I want to subscribe again. Although I only can get results with partial data, I tested the New Search and was very concerned about the changes in exact searching. Now they admit it is an error. Again, it was something that should have been recognized before putting it out to the public.

    For now, I WILL NOT recommend Ancestry.com or Family Tree Maker to newbies like I used to. Instead, I will refer then to Heritage Quest on our library site (accessible from home with library card number); the newspaper archives on our library sites (also accessible from home); and promote other online and offline sources. I do however suggest posting detailed queries on Rootsweb message boards (now Ancestry.com's) to attempt to find other researchers.

    I agree that Ancestry.com is spending far too much on marketing. Look at all the CNN ads! Yikes! How many serious researchers are they going to attract? It is clear that some new users who post are under the impression that Ancestry.com has a wealth of information about everyone. They obviously are taken in by the ads and have NO UNDERSTANDING of how to enter search criteria. Some of them may learn while others will drop their subscriptions at the end of the subscription...if they know/remember that they need to formally cancel it.

    They could hire a whole lot of great testers for the money they spend on the CNN advertising.

    Ultimately quality and good customer service will help a company grow more than a HUGE Advertising account. But, I guess Tim wants to count all those records people add to their trees as part of the statistics regardless of whether it is a duplicate of another tree or not. Eventually, there will be a whole lot of trees and fewer customers with their current strategy.

    -lhm

    ReplyDelete
  7. New Search results require too much scrolling. I tried BMD in UK. Names with middle names now take up three lines versus two.
    The date is compressed to fit into a window.
    The previous format was much better.

    ReplyDelete
  8. new search does not provide any improvement in scrolling large hit lists.
    A lit list of 137 in BMD with results set at 50 only has a selection of pages 1 and 2.
    whereas Old search has 1 2 and 3.

    I'd like to be able to jump around browse hit lists much better than that.

    To oversome the lousy 3 letters before wildcard I frequently search census with just first names and then forward to Letter blocks. A lengthy process when only a few initial pages are provided in the selection

    ReplyDelete
  9. I agree with Jerry Bryan - the Ancestry fuzzy search is useless.
    Instead of Fuzzy you would do better to improve the wildcard function.

    I would like to be able to search names with a ? in the second letter position

    ReplyDelete
  10. This is not quite the same issue but is search related.

    Ancestry.com now has some new census material for Florida, the problem is that their search page for it is broken and they do not seem eager to fix it. If you search for a family name (last name only nothing else) in a county where that family is common, it comes back with the results for 1885, only 1885. Try it and see. If it does not work complain to Ancestry.

    The State Archives sold them the film for the 1935 and 1945 State census as well as the bits of other year that the State has census data for. It should have been online around the first of the year. In the past you could search for the Mortality Schedule for the years 1850-1880 and it would bring up the schedules, now it does only the 1885, a step backwards!

    I have been trying to get them to fix this for 6 months with no luck. several emails from me and others have done nothing.

    Florida State Censuses, 1867-1945
    http://content.ancestry.com/iexec/?htx=List&dbid=7605&offerid=0%3a7858%3a0

    Ancestry keeps rushing to add more without completely testing what they add.

    After filling out a 42 question "you have been selected" survey, it is apparent that thay plan to add lots of extra charge services, fix the broken stuff before selling us anything else!

    Best to all, Ron

    ReplyDelete
  11. Having used the old searching and now the new one, I no longer find that Ancestry.co.uk is worth the subscription. Unless they sort this out people will walk! It is no longer possible to get back close matches it seems you get all (which is usually too much) or nothing.

    ReplyDelete
  12. Dear UK friend,

    A general statement like, "it is no longer possible to..." is not actionable. Since July, users have identified several real, serious errors which they have fixed. But you must post the exact steps for something that doesn't work before they can fix it. Here's an example:

    Steps with old search:

    * Click on the Search tab.
    * If Historical Records is not selected, select it.
    * Check the Exact matches only box.
    * Enter the name Benjamin Wiser.
    * Search.
    * Click on Massachusetts Town Birth Records.
    * Expected result: see the 5 children of Benjamin Wiser.

    Steps with new search:

    * Click on Search tab (or link).
    * Click on Show Advanced.
    * Result: list of results from all sorts of databases.

    With exact instructions, Ancestry can see what they have messed up. In my example, they realized they had dropped the ability to view the results summarized by category. They added that capability back in. And I think when the exact box is checked, they select that by default. Otherwise I as a user still have to learn to change the View from "Sorted by relevance" to "Summarized by category."

    -- The Insider

    ReplyDelete