Monday, June 28, 2010

Evidence Management Diagram Revisited

Last week I wrote about evidence management and the New FamilySearch Tree. The plan this week was to write about Ancestry.com Member Trees. I struggled as I wrote. I decided Ancestry.com has a piece of evidence management that isn’t represented in my model. It was time to revisit the evidence management diagram.

Here is how it has looked:

An old Evidence Management Diagram

Since I’m hardly an expert on genealogical methodology, my model draws heavily on experts. Elizabeth Shown Mills “Evidence Analysis: Research Process Map” has these basic components:

 The Evidence Analysis Research Process Map

From sources we draw information. From information we choose evidence. The proof of a conclusion lies in a careful analysis of the source, the information, and the evidence.

Upon this foundation, I drew upon my technical background for my contribution. What interfaces (technically, user interfaces and underlying objects) does a genealogy program need to implement this? Desktop genealogy programs already have interfaces for entering sources and for displaying individuals. I came up with two more: an evidence summary, and a conclusion entry interface. Here are the four interfaces juxtaposed beneath the evidence analysis components:

Evidence Analysis plus Evidence Management

The bottom row became the evidence management diagram shown at the beginning of this article. Some of the boxes are displayed more than once to communicate some technical stuff that I won’t bore you with.

Writing about Ancestry.com Member Trees, I realize there is another interface that can play an important role in evidence management. I include it in the new evidence management diagram:

 The Evidence Management Diagram

The circular nature of the new diagram evolved (revolved ;-) from the addition of the new Compare interface.

  1. To evaluate a potential source, we compare all the “facts” we believe about an individual with information in the source. If the comparison is favorable, we have identified a new source.
  2. Through a source interface, we enter a citation and other information about the source.
  3. We take information from the source and create an evidence summary.
  4. To help us make a conclusion, the analysis interface displays relevant evidence.
  5. Our conclusion becomes one of the “facts” displayed about a person.

While names may have changed, the function of the red, green, purple, and blue boxes remains the same as before.

What do you think?

  • Are the changes an improvement?
  • Is it easier to understand?
  • Does it meet the needs of newer users? Experienced users? Genealogy program software engineers?
  • Is the circular format appropriate?
  • Have I correctly applied industry terminology?
  • Do the interface names accurately reflect the function of the interface as explained here and in previous articles?
  • There are technical inaccuracies, to be sure. (For example, information comes from the source, not the source interface.) Are there inaccuracies that can be corrected such that the usefulness of the diagram increases?

I have come to depend upon your feedback during this series of articles. After you’ve had a chance to respond, I hope to have the stamina to go back and revise all the previous articles with the new diagram and the new terminology.

Thanks in advance.

With this new model, I am ready to take on Ancestry.com… Next time…

7 comments:

  1. The circle is good: It emphasizes that the search for evidence is never over. Even if you've completed a reasonably exhaustive search, carefully analyzed all of the resulting evidence, and written up a nice proof statement, you still need to be open to finding more evidence and having to go through the whole thing again.

    I'm really bothered, though, by the evaluation line: You say "To evaluate a potential source, we compare all the “facts” we believe about an individual with information in the source. If the comparison is favorable, we have identified a new source."

    I say, if the comparison is unfavorable, you've still identified a source; it's just conflicting evidence that needs to be resolved rather than supporting evidence that leaves you in your happy place. If it's irrelevant, it's still a source, just not one that sheds any light on the problem at hand. The important thing, though, is that if it appears to apply to the case in any way, then it's evidence and needs to be analyzed with the rest.

    I would add a "Search Interface" box to the circle, and follow it with the "Source Interface", then the "Compare Interface" (which I would rename "Relevance") and then "Evidence Summary" and so on.

    ReplyDelete
  2. Dear John,

    You're awesome, dude! I love the quote, "The circle...emphasizes that the search for evidence is never over." No conclusion is ever final.

    We are in agreement regarding conflicting sources. Sources that may one day prove useful in an indirect proof should be similarly handled.

    I can see that my wording doesn't clearly communicate that.

    Is the problem with the word compare? We can change the name of the interface if necessary. Personally, I think compare describes what we do when we decide if a source "appears to apply to the case in any way."

    So I think the ambiguity arises out of the word "favorable."

    I like your phrase, "appears to apply to the case in any way." For the truly anal, it might be a touch too loose.

    I'd like to also consider relevant, but I'm uncertain with your use of irrelevant. Certainly, all sources are sources. Is that all you're trying to say? Or is there a problem with using the word relevant?

    Thanks for your thoughts,

    -- The Insider

    ReplyDelete
  3. Assuming we find the perfect methodolgy to go from Source to Conclusion, the next question is how do we record the process in our genealogy software programs?

    I am using RootsMagic4 which allows you to create Notes for every Person, Family, or Fact. It is here that I record my assessment of the evidence and how I arrived at my conclusions.

    I am not convinced that my Notes need to be expressed in some "Evidence Management Template" but perhaps the professional genealogist would disagree.

    RootsMagic also allows you to create Comments for each Source but I use this feature to just comment on the Source itself.
    John Carruthers

    ReplyDelete
  4. Certainly I find the new diagram much clearer because it is / appears to be (delete as approp!) a diagram about processes, with outputs, inputs etc., whereas I was never wholly sure whether your first diagram was a data model or a process diagram. If it is a process diagram then the usual convention is to name them "verb noun" style and the "Compare Interface" might be labelled "Assess Relevance" (or "Assess Relevance of Source"). As John R says, whatever the outcome, it's still a source, it's just that there might be something that, at first glance, makes it such a no-no that it's not worth going much further. (E.g. it's a baptism that you thought might be your target but you discover a note added by the minister that says "buried next week". You'd want to record this if you're keeping all your research inside your FH program).

    I'd alter the names of later boxes similarly, e.g. "Record Source Details"; "Summarise Evidence"; "Analyse Evidence"; "Record Conclusions".

    "Person Interface" is not appropriate for 2 reasons in my view - firstly it's an IT system we're talking about so "interface" is taken as read, and secondly it might be a family, an organisation, a place, etc, that we're dealing with.

    No doubt others will have different names.

    (A personal comment at this point - I severely dislike "Information" being the output from "Source Interface" / "Record Source Details". I was always brought up that Data becomes Information only when it's relating to something of note. In that view, it's still just Data in Sources and doesn't become Information until after analysis has linked it to people. However, I got the impression that Information is the 'standard' word, so I might not win that!)

    Just to be awkward - shouldn't there be something about setting research objectives? Otherwise how can we assess relevance or analyse the data?

    Adrian

    ReplyDelete
  5. I'm only at what could be generously categorized as an intermediate-level genealogist, but have a background in the corporate research world (competitive intelligence & market research). It's been a very interesting series.

    I still like a linear format. The circular format seems to imply that each box has a 1:1 relationship to the next box. Since a single source can contain many pieces of information, you'd need branches off of the Source Interface for each item (i.e. I'd treat info about birth date on a death certificate differently than I would treat the death date).

    I'm also missing the step of 'state the hypothesis' in the circular loop. My shorthand explanation of a research process to non-researchers is basically (1) state the hypothesis, (2) collect & process sources/data, (3) analyze and (4) does it prove or disprove the original hypothesis. Where's that 1st step in your loop?

    Just my 2 cents...

    Pat M

    ReplyDelete
  6. "Evidence Management"
    What interfaces (technically, user interfaces and underlying objects) does a genealogy program need to implement this?

    I believe this depends on how you use your software. To me it is a collection of data. From this collection I can draw data that I can turn into information and eventually, evidence to answer various questions. Mills states that evidence is what is extracted from information in response to a question, we then analyze this evidence to draw our conclusions/proofs. So how do you manage something that changes with every question?

    It seems to me that the model implies we are doing our analysis out of context in advance of entering the data with the assumption appearing to be that only proven data is good. I think that conflicting or incorrect data has its place in our database of "facts," "events," or whatever your software calls them as long as it is documented as such. That way it shows you know about the incorrect data, analyzed it and and it states why you disregard it. I tend to document my conclusion or proof arguements in the "Notes" attached to the specific item it affects (fact, event, person, etc.

    I prepare research plans to focus my analysis or need for further research. This is an area where I think all desktop software is weak. I still use a word processor, although I may generate a timeline in my software and cut and past into the plan.

    There is a rudimentary evaluation of a source "record" to select it for your database. You wouldn't save all the John Smiths (records) in the 1900 US Census (source). You may use evaluation criteria (filters) such as, location, spouse, or age.

    Finally, I found you circle diagram very similar to the one used by Karen Clifford in her srticle "The Research Cycle," which can be found at http://www.genealogy.com/84_clifford_print.html

    ReplyDelete
  7. As noted in my post that has not appeared, Ancestry's "new Compare interface" goes with the purple box, evidence analysis.

    Of course when the user is working from Trees, there is scarcely ever any evidence to be had; all conclusions, nearly all the time.

    I agree with the commenters noting that there ought to be a question/quest/new-hypothesis section of the circle.

    ReplyDelete