This research was conducted by a group of independent astrologers with the aim of confirming or refuting the hypothesis that astrological forecasts are random guesses.
The objective of this work is to obtain a statistically significant result with a significance level of 5% or less, allowing us to refute or confirm the null hypothesis of the randomness of astrological forecasts.
The PDF version of this article is available here.
[toc]
Events Under Examination
In this study, it is important to define the concept of a "forecasted event." In real life, an event is the simultaneous occurrence of a vast number of elementary outcomes. For instance, a person being fired from a job assumes various conditions: being dismissed in written form, the order being signed at a specific minute on a certain type and quality of paper, the atmospheric pressure at that moment being a certain value, and so on. However, from an astrological standpoint, we forecast and subsequently measure only one aspect of this complex system—the individual's state of being fired or retained in the considered time interval. Other parameters are not part of the forecast and are not subject to verification.
It's also essential to note that by the occurrence of an event, we mean the direct encounter of the individual with the forecasted event. For instance, in the case of predicting dismissal, it's not the moment of signing the order or leaving the office but the moment when the person first learns about the dismissal.
Usually, the context of the question within which an astrologer makes a forecast clearly defines which observable parameter of life is subject to prediction. For example, if a woman inquires about the date of pregnancy, the astrologer specifies the date when the woman learns about the pregnancy. If the question is about the date of childbirth, it refers to the moment of the child's arrival, and so forth.
Terminological Basis
In the scope of this research, we'll be using the following definitions:
-
Measurable Parameter: This is a part of an event voiced in the forecast and can be objectively verified. For example, a forecasted event like "dismissal with an unexpectedly substantial compensation" implies two parameters—dismissal and compensation payout.
-
Elementary Outcome: These are discrete values that the measurable parameters can take. The discrete value of "true/false" for the occurrence or non-occurrence of an elementary outcome can be unequivocally verified.
It's fair to say that in astrological forecasts, there exist descriptions of outcomes whose unambiguous verification seems impossible. For instance, an astrological forecast might include qualitative descriptions such as "abundant/moderate precipitation," "significantly high/low temperature," "stout/slim individual." For instance, in the forecast "dismissal with an unexpectedly substantial compensation," there are three parameters—dismissal, compensation payout, and the amount of compensation (in the case of payout). Dismissal and payout can take two discrete and verifiable values of "true/false," whereas the compensation amount can assume 3 qualitative values—"substantial/moderate/small (relative to the current position)"—and doesn't appear straightforward to measure.
For the sake of purity in verification, we excluded forecasts from the sample that contained such qualitative descriptions of future events.
- Probability of Event Occurrence (also known as the probability of random guessing): This is the final probability of the occurrence of elementary outcomes, which can be dependent or independent of each other.
Research Conditions
Homogeneity of the Sample of Tested Astrological Methods
To maintain uniformity in the astrological tools under examination, we selected astrologers who employ the same forecasting method (known as traditional 17th-century horary astrology1 and horoscope validation2 before making a forecast). These individuals are not novices, actively practice astrology, and solely engage in event forecasting.
Sample Selection of Forecasted Objects
We did not impose restrictions on the selection of forecasted objects, assuming that horary astrology methods are equally effective across all spheres of human life. The results of their application do not depend on the object of prediction, much like measuring the weight of stones using identical scales regardless of the stones' shapes or colors.
It's worth noting that there are two types of questions where astrologers believe discrepancies between forecasts and reality are more common. These involve descriptions of where a client will soon find a lost item and determining the gender of a child. To maintain experimental purity, these questions were excluded from the analysis.
Forecast Requirements
The study was based on both fulfilled and unfulfilled astrologer forecasts, which were made and documented strictly before the predicted events occurred.
The primary forecast requirements were:
- Objective verifiability of predicted events. Forecasts should not include descriptions of subjective experiences or abstract immeasurable quantities such as "he will love you," "there will be improvements," "a period of instability," and similar terms. Nor should they contain ambiguously verifiable factors like "high income," "reduced price." All forecasts had very clear formulations, including descriptions of the verifiable details. For instance, in response to a question like "how will relationships develop," an astrologer might forecast that exactly 2.5 weeks later, the client will discover his wife's infidelity in their home, followed by a divorce.
- The date of the forecast should precede the date of the event. Descriptions should not be modified, adjusted, or interpreted retroactively in favor of an event that subsequently occurred.
- An obligatory criterion was the presence of both fulfilled and unfulfilled forecasts within the period, as well as the absence of concealing unfulfilled forecasts.
There were no requirements regarding the verbal formulation of the forecast. Forecasts could be binary phrases like "event will occur/will not occur" or descriptions of future event details like "on November 25 under these conditions, such-and-such unexpected event will happen."
Feedback on Fulfilled and Unfulfilled Forecasts
In actual practice, only a portion of clients provide feedback on the occurrence or non-occurrence of the predicted event. We have no grounds to assume that clients are equally inclined to provide feedback to an astrologer in the event of occurrence or non-occurrence of the predicted event.
To eliminate this uncertainty in the examined sample, two astrologers made forecasts exclusively for themselves, their close relatives, and family members, having 100% feedback on the results of their forecasts.
We used two samples for the study. The first contained all forecasts made within a specific period, where astrologers documented their forecasts and later received confirmation or refutation.
Subsequently, we excluded all forecasts that could be considered self-fulfilling and retained only those where the outcome of the forecast was independent of the person receiving the forecast. These forecasts constituted the second sample.
Assumptions and Approximations
Assumptions Regarding the Assessment of Event Occurrence Probabilities
In calculating the probabilities of each elementary outcome in an event, we made rough estimates based on the available information. For example:
- If the predicted outcome was extremely unlikely (e.g., forecasting snowfall in Sri Lanka on a specific date - which wasn't included in our sample), we attempted to estimate the probability of the outcome by crudely dividing the number of low expected outcomes by the number of all possible outcomes. Similarly, we handled the most probable outcomes.
- In several cases, we only had qualitative probability estimates of event occurrence, such as "less likely" or "extremely unlikely." In such cases, we created a simple model; for instance, we took an interval from 40% to 50% for "less likely events" and an interval from 1% to 30% for "extremely unlikely events," then considered three values within each interval. We followed a similar process for events considered "more likely" and "almost inevitable," generating three calculation results. Clearly, this is a very approximate reflection of reality, but we employed this model due to the absence of more reliable information. The research task was to determine if there was a consistent result among all three calculations and if there were discernible signs of astrological forecasting.
- When the outcome seemed equally probable in a given context (e.g., the chance of winning a court case when lawyers' assessments of each side's positions are roughly equal), we considered the probability of such an outcome to be approximately 50%.
- When forecasting the timing of events, we looked at the deviation between the forecasted timing and the confirmed actual timing. For example, if a forecast promised that an event (a person's encounter with a predicted object) would occur in 21 days, and it happened in 23 days, we approached it as follows: we took the deviation from the real date (in our example, it's 2 days) and calculated the total probability of falling within the $\pm2$ days interval around the actual event date. For instance, if the expected period for the event occurrence was 2 months in context, the probability of landing within a 4-day interval would be $4/61 \approx 6.5\%$ (in this research, we considered the event occurrence dates as equally probable).
- In the case of multiple elementary outcomes, we estimated the overall probability of their realization. For instance, while predicting a termination date, we assessed the product of the probability of being dismissed in principle and the conditional probability of being dismissed on a specific date. This provided us with the probability of randomly guessing this event in the forecast. Similar estimations were made for all forecasts in the sample.
Problem Statement
We have a dataset of $n$ astrological forecasts, of which $k$ turned out to be accurate. The probabilities of random guesses for these events constitute an array of values $P_1...P_n.$
The null hypothesis was that the probability of guessing the result $k$ times out of $n$ forecasts is not a significant value at a $\le 5\%$ significance level and does not prove any regularities in the forecasts. To test the null hypothesis, we used the Poisson binomial distribution, which shows the probability of guessing $k$ events out of $n$ attempts with probabilities of individual guessing $P_1...P_n$ respectively:
Here, $F_k$ is an array of possible samples of $k$ elements from the set of integers ${1..n}$ of length $C^n_k,$ $A$ represents an element of the array $F_k,$ and $i$ iterates over the indices within the sample $A.$ For example, with $n=5$ and $k=3,$
and so on, repeated $C^5_3$ times.
Next, $A^c$ denotes the complement of sample $A$ with respect to the set of integers ${1..n}.$ For instance, if $A={1, 2, 3},$ then $A^c={4, 5}.$ Index $j$ iterates over values ${4, 5}$ - and this holds for each $A\in F_k.$
The research task was to calculate the quantity $P(X\ge k) = 1 - \sum\limits_{i=1}\limits^{k} P(X=i)$ and compare it with a $5\%$ significance level to test the null hypothesis.
Research Results
Sample 1
Our first set of forecasts consisted of predictions made by astrologers for themselves, close relatives, and family members, including 100% feedback on both fulfilled and unfulfilled forecasts. It is represented in the following table of forecasted events, forecast outcomes, as well as:
- Qualitatively assessed (Pe) probability of random guessing
- Taken into account (P1, P2, P3) probabilities of random guessing. Here, P3 represents the maximum possible value of random guessing, where the occurrence of unlikely and nearly impossible events is estimated as 1/2, while the probability of more likely or almost inevitable events is always set at 99%.
| ID | Forecasted Event | Status | Pe | P1 | P2 | P3 |
|---|---|---|---|---|---|---|
| 1 | Loss in court | Confirmed | $\approx90\%$ | $90\%$ | $90\%$ | $90\%$ |
| 2 | Removal from military service in the midst of conscription | Confirmed | $\le 50\%$ | $40\%$ | $45\%$ | $50\%$ |
| 3 | Loss in court | Confirmed | $\approx90\%$ | $90\%$ | $90\%$ | $90\%$ |
| 4 | Encounter with a combat zone in Israel in 8.5 days without life-threatening situation | Confirmed | $\approx 3.5\%$ | $3.5\%$ | $3.5\%$ | $3.5\%$ |
| 5 | Expected visa acquisition | Confirmed | $\gg50\%$ | $70\%$ | $85\%$ | $99\%$ |
| 6 | Inability to take the specified flight | Unfulfilled | $\ge 50\%$ | $50\%$ | $55\%$ | $60\%$ |
| 7 | Notification of document readiness exactly in 5 hours | Confirmed | $\approx4.8\%$ | $4.8\%$ | $4.8\%$ | $4.8\%$ |
| 8 | Expected salary increase within six months | Unfulfilled | $\approx50\%$ | $50\%$ | $50\%$ | $50\%$ |
| 9 | Trip absence to the specified location in the indicated period | Confirmed | $\approx50\%$ | $50\%$ | $50\%$ | $50\%$ |
| 10 | Denial of residency permission in another country | Confirmed | $\approx50\%$ | $50\%$ | $50\%$ | $50\%$ |
| 11 | Positive decision on job acceptance | Confirmed | $\ge50\%$ | $50\%$ | $55\%$ | $60\%$ |
| 12 | Introduction of QR codes in transportation amid epidemic | Unfulfilled | $\ge50\%$ | $50\%$ | $55\%$ | $60\%$ |
| 13 | Unimpeded departure from the country by year-end (there was a chance of delay due to uncertainty in the appointment of administrative work) | Confirmed | $\approx50\%$ | $50\%$ | $50\%$ | $50\%$ |
| 14 | Child's lack of contact with the father in the specified period after the mother's request for resuming contact | Confirmed | $\approx50\%$ | $50\%$ | $50\%$ | $50\%$ |
| 15 | Non-delivery of the apartment from the developer within the agreed timelines | Confirmed | $\approx50\%$ | $50\%$ | $50\%$ | $50\%$ |
| 16 | Absence of new clients in the specified period | Confirmed | $\approx50\%$ | $50\%$ | $50\%$ | $50\%$ |
| 17 | Victory in court | Confirmed | $\ge 50\%$ | $50\%$ | $55\%$ | $60\%$ |
| 18 | Husband's decision to extend the contract and not return home | Confirmed | $\approx50\%$ | $50\%$ | $50\%$ | $50\%$ |
| 19 | Sale of a personal item to a potentially interested buyer | Confirmed | $\gg50\%$ | $70\%$ | $85\%$ | $99\%$ |
| 20 | Lack of expected repairs from the developer after filing a complaint | Confirmed | $\approx50\%$ | $50\%$ | $50\%$ | $50\%$ |
| 21 | Loss in court | Confirmed | $\ge 50\%$ | $ 50\%$ | $55\%$ | $60\%$ |
| 22 | Successful sale transaction of the apartment to the specified buyer | Unfulfilled | $\approx50\%$ | $50\%$ | $50\%$ | $50\%$ |
| 23 | Renting out the apartment in the specified period | Confirmed | $\approx50\%$ | $50\%$ | $50\%$ | $50\%$ |
| 24 | Absence of military confrontation | Unfulfilled | $\ge50\%$ | $50\%$ | $55\%$ | $60\%$ |
| 25 | Absence of positive results in husband's oncological tests | Unfulfilled | $\approx50\%$ | $50\%$ | $50\%$ | $50\%$ |
| 26 | Unexpected car engine breakdown on a trip without life-threatening situation | Confirmed | $\approx12\%$ | $12\%$ | $12\%$ | $12\%$ |
| 27 | Absence of damage to fragile items during transportation | Confirmed | $\ge 50\%$ | $50\%$ | $55\%$ | $60\%$ |
Sample 2
Next, we excluded forecasts in which individuals knowledgeable about the forecast could have influenced the outcome and potentially unconsciously created circumstances for a known outcome. These forecasts turned out to be forecasts #4, 8, 9, 16, 18, which we excluded from the sample. We considered the remaining sample to be highly reliable. It contained 22 forecasts, 17 of which turned out to be accurate.
The Poisson distribution for this sample looks as follows:
We also took a control group of random individuals and asked them to guess the answers to the same questions that the astrologers considered. For instance, a question might have been: "Guess whether a couple will be intimate after a breakup, and if yes, how many days afterward?"
On average, the control group guessed 10 correct answers out of the 22 questions. The group of astrologers provided 17 correct answers to the same 22 questions.
In the non-astrologer control group, the P-values for random guessing 10 out of 22 attempts for the dataset {P1, P2, P3} are 79%, 87%, and 93% respectively. This confirms randomness in guessing.
In the astrologer group, the P-values for random guessing 17 out of 22 attempts for the dataset {P1, P2, P3} are $0.7\%,$ $1.5\%,$ and $2.9\%$ respectively, which are below the $5\%$ threshold value.
The table below summarizes our results
| Group | Number of Correct Guesses | P-value | Threshold |
|---|---|---|---|
| Random People | 10 out of 22 | 79% - 93% | 5% |
| Astrologers | 17 out of 22 | 0.7% - 2.9% | 5% |
Verification of the Accuracy of Astrological Prediction of Timings
Method for Assessing the Probability of Randomly Guessing Timings
When estimating the probability of guessing a timing purely by chance, we proceeded from the following considerations:
- We determined the period within which the predicted event was expected to occur with 100% certainty. This period was predetermined in the context of the situation under consideration. For example, if it was expected that the dismissal would occur within six months, we set the period at 180 days.
- In our model, we operated on the postulate of the independence of the event's occurrence from the date of the event. In the case of dismissal, we considered being dismissed on a specific date to be no more or less likely than on any other within the given period.
- We considered guessing the period successful if the guessed timing deviated from the actual timing by no more than 10% of the duration of the period in which the predicted event was supposed to occur.
- If the astrologer's forecast deviated from the confirmed date by X days, we considered the sum of the probabilities of randomly guessing any day within the interval of $\pm X$ days around the predicted date as the probability of randomly falling into that interval.
Skeptical Assumptions
Assumption Regarding the Probability of Event Occurrence
Although the probability of randomly guessing the occurrence of events was not 100%, for a more rigorous analysis, we assumed that all predicted events were inevitable, and their probability of occurrence was 100%. However, we focused only on the part of the forecast related to timings.
Assumptions about Predictions with Unknown Outcomes
During the control period, four astrologers participating in the study made 203 forecasts. In 101 instances, the astrologers received feedback from clients regarding the outcomes of their forecasts. Out of the 203 forecasts, we randomly selected 64 forecasts that met the aforementioned requirements. Among these 64 forecasts, 10 included specific timings.
In 49.7% of cases, clients didn't provide feedback on the outcomes of any forecasts, including those with specified timings. We assume that clients didn't provide feedback because all these forecasts didn't materialize.
Thus, we assume that for every 10 forecasts with known outcomes, there are no more than an additional 5 unfulfilled forecasts.
We also assume that the probability of randomly guessing timings in forecasts with unknown outcomes does not exceed the maximum probability of guessing timings in forecasts where the outcome is known.
Assumptions Regarding the Maximum Estimation of Timing Guessing Probability
The probabilities of guessing the timing of events in 10 forecasts vary. For example, the probability of randomly hitting a three-day interval within a month is 1/10, while the probability of guessing a specific hour within two days is 1/48, and guessing a specific day within six months is 1/180. In the most skeptical assumption, we assume that the probability of randomly guessing timings in astrological forecasts in our sample does not exceed the maximum value (in our example, 1/10).
In this scenario, we can apply a simple binomial distribution to estimate the maximum probability of k random timing guesses in n forecasts.
If the maximally estimated P value of k random timing guesses in n forecasts does not exceed a 5% significance level, we will have sufficient grounds to unambiguously reject the null hypothesis and accept the alternative—thus accepting that astrology is indeed capable of predicting timings.
Research Findings
We had the following sample of astrological forecasts with specified timings at our disposal. In the table below, PE stands for the probability of randomly guessing the event, and PT stands for the probability of randomly falling within the interval between the actual and predicted events.
Random Guessing of k Times in n Forecasts Not Exceeding a 5% Significance Level
If the likelihood of randomly guessing k times in n forecasts does not exceed a 5% significance level, we will have sufficient grounds to unequivocally reject the null hypothesis and accept the alternative, which means accepting that astrology is indeed capable of predicting timings.
Research Results
We had the following sample of astrological forecasts with specified timings at our disposal. The table below indicates PE - the probability of randomly guessing the event, and PT - the probability of randomly falling within the interval between the actual and predicted events.
| Forecast | Outcome | Δ | Pe | Pt | |
|---|---|---|---|---|---|
| SMS from a government agency will arrive in 5 hours within the next 21 hours | SMS arrived in 5 hours | 0 hours | 100% | 4.76% | |
| Intimate closeness with a man will happen in 4 days within the next month | Closeness occurred in 3.5 days after the forecast | ±0.5 days | <100% | 6.67% | |
| Female cycle should resume in 18 days within the next month | Cycle resumed in 18 days | 0 days | <100% | 3.33% | |
| Next order will happen in 17 days within three months | Order received in 17 days | 0 days | 100% | 1.11% | |
| Death of a terminally ill patient will occur in 28 days within the next 5 months | Death occurred in 28 days | 0 days | 100% | 0.66% | |
| Son will fall ill, it will happen in 11 days | Illness occurred in 11 days | 0 days | <100% | 3.33% | |
| Expected female cycle will start in 13 hours within the next week | Cycle started exactly in 13 hours | 0 hours | <100% | 1.79% | |
| Receiving money will happen within 3 weeks. It happened in 5 days | Money received in 5 days | 0 days | 100% | 4.76% | |
| Departure from the country will occur. It was predicted in 7 days within two weeks | Not realized | <100% | 7.14% | ||
| Department head will leave within six months, specifically in 32 days | Head left in 32 days | 100% | 0.56% | ||
| Surgery will occur in 2 hours within two days | Not realized | 100% | 2.08% | ||
| Random Forecast #1 without feedback | Considered unfulfilled | $\le100\%$ | $\le10\%$ | ||
| Random Forecast #2 without feedback | Considered unfulfilled | $\le100\%$ | $\le10\%$ | ||
| ... | ... | ... | ... | ||
| Random Forecast #10 without feedback | Considered unfulfilled | $\le100\%$ | $\le10\%$ | ||
| Random Forecast #11 without feedback | Considered unfulfilled | $\le100\%$ | $\le10\%$ |
In the most skeptical model, we assume that the probability of randomly guessing the timing of an event is no more than 10%. The Poisson distribution for our sample looks as follows:
The control group of random individuals, on average, guessed 0.3% of the correct answers (there was no instance where everyone guessed at least once). Meanwhile, the group of astrologers provided at least 9 correct answers out of 22 forecasts.
Using the Newton binomial distribution formula for P=10%, we derive that:
- For the group of random individuals, the P-value cannot be lower than 66%.
- For the group of astrologers, the P-value cannot exceed 1.4E-2%.
The table below summarizes our results for guessing event timings:
| Group | Number of Correct Guesses | P-value | Threshold |
|---|---|---|---|
| Random People | 0.3 out of 22 | > 66% | 5% |
| Astrologers | 9+ out of 22 | < 1.4E-2% | 5% |
Conclusions
As observed, the P-value of randomly guessing an event's outcome, as well as the maximum possible P-value for guessing event timings:
- Exceeds 60% for individuals not using astrological forecasting methods.
- Is below the 5% level of statistical significance for individuals employing astrological methods.
Hence, we lack sufficient evidence to consider astrological forecasting as random guessing.
We believe that these results provide a basis for conducting a more precise experiment to verify astrological forecasting, where:
- The probability of the predicted event occurring will be assessed more accurately, and
- Conditions can be created for independent measurement of the occurrence of predicted events and minimizing the participants' influence on the event's outcome.
We also believe that this study demonstrates the following important facts about astrology:
- There are specific areas within astrology suitable for scientific verification. These areas require the keen attention of the academic community and further experimental testing under more rigorous conditions.
- Due to the presence of specific predictive branches within astrology that can be subject to verification, not all astrology can be unequivocally considered pseudoscience, although many of its branches do not meet the requirements of a scientific discipline.
In Conclusion
The group of astrologer-forecasters conducting this study invites collaboration from all representatives of the scientific community for joint verification of forecasts under controlled conditions, with confirmation of results by an independent group of observers.
-
Horary astrology is a branch of predictive astrology dealing with detailed predictions of specific life situations. All participating astrologers in this study follow the techniques of the British astrologer W. Lilly, who predicted the Great Fire of London and was known for many accurate forecasts, including locating thieves, where constables were sent and found the missing item. ↩
-
Horoscope validation is a technical procedure to ascertain the ability to make a prediction. This procedure, known as "checking the radicality of a chart," is performed by an astrologer before applying forecasting techniques. ↩
