Bayes
INTRODUCTION
I have read through many of the files here on the Crucible regarding UFO's and the possible involvement of the United States government with the same. Many of the documents ( such as the statement by John Lear and the Fenwick interviews) make a number of claims, but seem to offer little data to support those claims. What data is offered seems inconclusive to me. With the scarcity of data on one hand and a number of claims on the other hand, I am faced with a dilemma.
I can reject the arguments put forth by Lear and others that the U.S. government is involved with UFO's. To reject thes arguments I must dismiss some evidence that is both plausible and has no other explanation. I find this option undesireable because some of the evidence supports the claims of Lear et al and is hard to refute.
My alternative is to accept the claims of U.S. government involvement with UFO's. To accept these arguments I must accept some statements that have little supporting evidence. I find such leaps of faith distasteful.
What other choices do I have? As I see it, I can use an existing technique for examining the claims and the evidence supporting them. That technique is Bayesian analysis. If we convert the Lear statements into hypotheses, we can then apply Bayes to the data. This process involves several steps.
STEP 1
The only requirement for the hypotheses is that they be mutually exclusive (one hypothesis can't encompass another) and collectively exhaustive (taken together, the hypotheses account for all possible explanations).
For example, the basic argument put forward by Lear is that the U.S. government has had contact with UFO's since the late 1940's and is not telling the truth about its involvement. I would break this into several hypotheses:
- The U.S. government has had contact with UFO's, is providing no accurate information on its activities, and is producing disinformation on the subject.
- The U.S. government has had contact with UFO's, is providing some accurate information on its activities, and some disinformation on the subject.
- The U.S. government has had contact with UFO's and is providing totally accurate information on its activities.
- The U.S. government has had no contact with UFO's, is providing no accurate information on its activities, and producing disinformation on the subject.
- The U.S. government has had no contact with UFO's, is providing some accurate information on its activities, and some disinformation on the subject.
- The U.S. government has had no contact with UFO's and is providing totally accurate information on its activities.
I think these six hypotheses are independent of one another (mutually exclusive) and cover the range of explanations (collectively exhaustive). Would anyone care to add to, modify, or replace these hypotheses?
STEP 2
Now that we have some hypotheses, we must make an initial assessment of their accuracy. The hypotheses must each be assigned a value between zero and one. The sum of the values for all of the hyotheses must equal one. [If you aren't familiar with Bayes, most textbooks on statistics have a section on it.] These values are then used with the incoming data.
If you want to work on this yourself, use a columnar worksheet (paper) or a spreadsheet (computer). Assign each hypothesis on a row of the sheet. In the first column to the right of the hypothesis, put your initial value. Set aside the next column for your first piece of data.
STEP 3
With initial hypotheses in hand, we can now take each piece of data and compare it to each hypothesis. We assign a value between zero and one to the data for each hypothesis. A value of zero for a given piece of data means that it absolutely denies a hypothesis. A value of one means that it absolutely supports a hypothesis. As you can see, very few pieces of data will fit either extreme. Instead, most data falls in between. [An example of a "one" value piece of data might be the President of the United States saying on national television that the U.S. government has been in contact with EBE's and that until now the government has been lying about it. This would rate a 1.0 for Hypothesis 1 above and a zero for Hypothesis 6.]
With six hypotheses, each datum must be evaluated six times and assigned six value (once for each hypothesis). On your worksheet (spreadsheet) put the value you have chosen into the column to the right of the initial value (as mentioned in Step 2 above). Multiply the initial value (Column 1) by the new value (Column 2) and place the product in the next column (Column 3). Add up the numbers in Column 3 and put the sum at the bottom of the column. [As you can see, a spreadsheet becomes handy very quickly.] Now divide each of the numbers in Column 3 by that sum at the bottom of the column and place the quotient in Column 4. What you should have should look something like this:
Hypotheses Initial Datum Product Revised
Value One Value
Hyp 1 0.2 0.4 0.08 0.24
Hyp 2 0.3 0.5 0.15 0.44
Hyp 3 0.1 0.2 0.02 0.06
Hyp 4 0.1 0.3 0.03 0.09
Hyp 5 0.2 0.1 0.02 0.06
Hyp 6 0.1 0.4 0.04 0.12
___ ____ ____
SUM 1.0 0.34 1.01** [Note round-off error. This sum should also equal 1.0]
This process can be continued for each new piece of data, using the revised product of the previous datum as the starting value for the next datum.
SUMMARY
I have participated in and led group problem-solving efforts with these techniques. Bayesian analysis is particularly useful for this type of problem. I can set up this sort of spreadsheet in either Lotus 1-2-3 (.WKS) or Microsoft format (SYLK). I think Tom will welcome this sort of exchange on the Crucible. Let me know if you are interested in helping.
I think this approach has considerable merit for the type of problems that are presented by the Lear/Krill/Fenwick statements. I welcome any individual or group efforts to isolate and evaluate the data available. Without the sort of approach I have described, I believe no serious assessment and cooperation is possible. Ufology will continue to spin its wheels with inconclusive data and unproveable theories.
- Bill Badger
26 Feb 89
The method I described previously is intended as a tool for evaluating how consistent various data are with a given set of hypotheses. It is not an evaluation tool for the data itself. Data inputs must be accurate and reliable, otherwise you are likely to get garbage.
For example, take President Reagan's remarks in Dec 1985 about, "Well, I don't suppose we can wait for some alien race to come down and threaten us...." Since this remark was widely reported, we can take it as both accurate (it reflects what Reagan said) and reliable (checking it from several sources gives the same answer). The issue then is consistency with our hypotheses (from BAYES.TXT).
- Hypothesis 1: US gov't contact, no disinformation. Reagan's remarks are very inconsistent (20% correlation).
- Hypothesis 2: US gov't contact, some disinformation. Reagans remarks are very consistent (80% correlation).
- Hypothesis 3: US gov't contact, all disinformation. Reagan's remarks are fairly consistent (60% correlation).
- Hypothesis 4: No US gov't contact, no disinformation. Reagan's remarks are fairly consistent (60% correlation).
- Hypothesis 5: No US gov't contact, some disinformation. Reagan's remarks are somewhat consistent (40% correlation).
- Hypothesis 6: No US gov't contact, all disinformation. Reagan's remarks very inconsistent (20% correlation).
Let's apply these judgements to our model (I picked the initial values for the sake of argument, not because I necessarily endorse them).
Hypotheses Initial Datum Product Revised
Value One Value
Hyp 1 10% 20% 2% 3.45%
Hyp 2 30% 80% 24% 41.38%
Hyp 3 25% 60% 15% 25.86%
Hyp 4 20% 60% 2% 20.69%
Hyp 5 10% 40% 4% 6.90%
Hyp 6 5% 20% 1% 1.72%
TOTAL 100% 0.58
