Tuesday, August 30, 2011

FSI My Accuracy

“My Accuracy” helps volunteers doing FamilySearch Indexing see how well their work survives the arbitration process. The feature was introduced in July. Your accuracy report appears as a new tab in the bottom right corner of the indexing start page. Your accuracy score reflects batches arbitrated in the last 90 days.

My Accuracy, a new FamilySearch Indexing feature

Click Review Batches to see a list of those batches. The Indexing application opens a browser page. Log in to see the list. Each line contains the index date, project name, description, accuracy score, and arbitration date.

My Accuracy's list of recently arbitrated batches

Click a project name

The review page looks similar to the indexing application. The field list at the bottom shows your values and, for those different from yours, the arbitrator’s values. The summary tab in the lower-right shows the accuracy calculation for the batch. In the example below, the batch contained 304 fields and the arbitrator agreed with 299 of my entries. Dividing the former into the latter yields 98%.

image

My Accuracy is provided as a tool that invites learning. In the example above, my entries differed from the arbitrators in 5 instances.

  1. imageI put “Carmino,” the arbitrator changed it to “?”. That is an obvious arbitrator error as a question mark signifies one illegible character. A more appropriate arbitrator response might have been “Ca*i*”, as these letters are quite obvious.
  2. imageI put “Grange,” the arbitrator changed it to “Orange.” This is an obvious handwriting interpretation error on my part. The O doesn’t look at all like a G. Both G and O occur elsewhere in the image. By using My Accuracy, I’ve learned that I am not being as careful as I think I am.
  3. imageI put “Cambridge” as the town, the arbitrator deleted Cambridge and left it blank. That is an arbitrator error. The town appeared on the back of the card and apparently the arbitrator hadn’t read the instruction to check there. In the arbitrator’s defense, FamilySearch has made it extremely difficult to learn the instructions for an indexing project. There are no less than six places that must be consulted to get all the instructions.
     
  4. imageI put 1863, the arbitrator changed it to 1865. Upon further review, I agree with the arbitrator. It looks like 3 was overwritten with a 5. There’s a case for specifying 186?. The problem with indexing it as 186? is, you’ve guaranteed a non-match for someone who searches for either 1863 or 1865. I would rather index this as either 1863 or 1865 and have a 50% probability of a match. (It is too bad that the search system can match ? to 3 or 5, but can’t match 3 or 5 to ?.)
  5. imageI left the state blank, the arbitrator changed it to Vermont. I see now that Vt appears after Shelburne, the town name.

If you feel the arbitrator was mistaken, click the Feedback link next to the arbitrator’s value. This opens a popup with a Please Review checkbox that you can check if you feel your original value. Unfortunately, FamilySearch doesn’t allow you to enter an explanation, which could save the reviewer a lot of time tracking down the town name on the back, or the state abbreviation in the corner. Without an explanation, the reviewer is in much the same situation as the arbitrator.

Actually, the instructions don’t state that FamilySearch will review the value. What action will FamilySearch take? In 90 days somebody remind me and we can check the published collection. Vermont marriage of Grace Phelps and Albert Martin, 8 September 1859. We’ll check the bride’s father’s first name. Will it be “?” which the arbitrator specified? Will it be “Carmino,” as I specified? Will it be “Cassius,” as suggested by Ancestry.com tree contributors? Or will it be something else entirely?

After many months we’ll finally find out what FamilySearch does when you check the Please Review box.

Or tomorrow I I could ask the FamilySearch Indexing product manager…

2 comments:

  1. I have complained forcefully to FamilySearch Indexing about not being able to give reasons for disagreeing with the arbitrator. It seems silly to not allow this. Perhaps you could use your contacts to encourage them to provide for this in the future, hopefully the near future.

    I also wondered to them whether anyone really looks at the instances where the Please Review box is checked. Here is the relevant part of their reply: "Thank you for contacting FamilySearch. We appreciate your feedback on My Accuracy. We are pleased that you are finding the feature useful. We know that it is frustrating when the arbitrators do not follow the field helps and project instructions and updates. We ask that you continue to send feedback when you disagree with their findings. These are reviewed by the My Accuracy project team and taken seriously. At this time there is no way to see the final results or to interact directly with the arbitrators." So it appears they actually do have a "My Accuracy project team" that reviews our disagreements with arbitrators. I wonder what the members of that team would say to having comments explaining the reason for disagreement. I bet a very high percentage of them would want that kind of feedback.

    ReplyDelete
  2. A bit late in adding to this theme - but whilst the 2 indexers/arbitration scheme sound good . . . in practice the arbitration is AWFUL!

    I live in England and am eager to help with the ongoing 1871 census project . . . was interesting to see an entry for he town I live in (Newent, Gloucestershire). . . was a little difficult to read but being familiar with these place names was definitely "Newent" and I entered it as such . . . bearing in mind the instructions to indexers . . . "If the town or city was misspelled, spell it correctly."
    Needless to say, the arbitrator has changed this to "Newertt" - of course thee is no such place. I find on any 1871 batch the arbitrator invariably changes some my entered place names to a totally nonsense interpretation of the place name . . . and must be happening on the majority of 1871 batches. The end result if not reformed, will be a joke - or laughing stock full of totally incomprehensible nonsense place names.

    When entering any difficult to decipher place name I always check it against a British Isles Gazetteer . . . why cannot arbitrators be instructed that if they change an indexers entry that the arbitrators version should be checked against a Gazetteer of British place names? I use and would recommend . . .
    http://www.gazetteer.co.uk
    ===Stewart

    ReplyDelete