mark_l_thompson - Network

Then I asked it: "How would you evaluate the work of this author in terms of its originality and skill?"And the answer was so...

Then I asked it: "How would you evaluate the work of this author in terms of its originality and skill?"

And the answer was so complimentary that I think I'll just go to bed now with that bit of generative flattery on my mind.

More later.

All in all, while few of the observations & suggestions were surprising, they weren't necessarily obvious, either. I'll...

All in all, while few of the observations & suggestions were surprising, they weren't necessarily obvious, either. I'll definitely have to consider how such a tool might be used to evaluate drafts of my own work as well as that of others. (Grading, anyone?)

Ultimately, Claude concluded with a generous assessment that summed up its previous remarks while also extending them:"The most...

Ultimately, Claude concluded with a generous assessment that summed up its previous remarks while also extending them:

"The most valuable contribution may be the framework itself, which could be developed into a more robust analytical tool with additional statistical analysis and historical verification. The identification of specific swing states within the 13 as key indicators provides a practical application for current electoral analysis."

But it was also critical in an appropriate way, too:"The author's transparency about methodological limitations and data...

But it was also critical in an appropriate way, too:

"The author's transparency about methodological limitations and data verification issues is commendable, though these same issues somewhat limit the current utility of the findings."

Finally, Claude offered a balanced & generous "Overall Assessment," that included the following:"The analysis presents an...

Finally, Claude offered a balanced & generous "Overall Assessment," that included the following:

"The analysis presents an intriguing hypothesis about the predictive power of the original 13 states in presidential elections. While the methodology needs more rigor and independent verification, the core insight about strong correlations between 13-state and national results merits further investigation."

7) Value Proposition offered a balanced & positive evaluation while the suggestions in 8) Future Research Opportunities were helpful. In...

7) Value Proposition offered a balanced & positive evaluation while the suggestions in 8) Future Research Opportunities were helpful. In the latter, Claude suggested:

"This could be expanded by:
- Analyzing state-by-state correlations over time
- Examining demographic changes in the 13 states
- Studying the impact of third-party candidates
- Developing a more robust statistical model
- Investigating causative factors for the correlations"

The next section turned from critique to advice, again useful if not altogether surprising:"6. Ways to Strengthen the AnalysisThe...

The next section turned from critique to advice, again useful if not altogether surprising:

"6. Ways to Strengthen the Analysis
The research could be improved by:
- Independently verifying historical election data
- Calculating specific correlation coefficients
- Analyzing demographic shifts over time
- Considering the impact of changing voting laws
- Including a more detailed methodology section
- Providing raw data and calculations
- Conducting statistical significance tests"

The experiment wasn't trying to do all of those things, but those were relevant criticisms. Next followed a series of related issues,...

The experiment wasn't trying to do all of those things, but those were relevant criticisms. Next followed a series of related issues, again quite relevant:

"5. Methodological Concerns
- Heavy reliance on LLMs for historical data processing
- Limited discussion of data sources
- No mention of controlling for changing population distributions
- Doesn't address changes in voting rights/eligibility over time"

Then it started to do offer some more interesting -- and more critical -- observations, namely:"4. Limitations and Potential Issues-...

Then it started to do offer some more interesting -- and more critical -- observations, namely:

"4. Limitations and Potential Issues
- Relies heavily on AI-processed data without independent verification
- Author admits they haven't double-checked Claude's numbers
- No statistical measures of correlation provided
- Limited discussion of demographic/social changes over time
- Doesn't address potential sampling bias (13 states may not be representative)"

Claude divided its response into eight sections, beginning with 1) Methodology 2) Key Findings & 3) Strengths of the Analysis, each of...

Claude divided its response into eight sections, beginning with 1) Methodology 2) Key Findings & 3) Strengths of the Analysis, each of which more or less summarized existing statements in the posts. But it did a good job of identifying key points from this basis.

The prompt I gave it was: "Evaluate the methods, arguments, and analysis of the following piece of text" and then I pasted in all...

The prompt I gave it was: "Evaluate the methods, arguments, and analysis of the following piece of text" and then I pasted in all but the last couple of posts from the original thread here on Mastodon.

Last week I wrote a series of short posts about my idea for a "13 States Index" to analyze the outcomes of past US presidential...

Last week I wrote a series of short posts about my idea for a "13 States Index" to analyze the outcomes of past US presidential elections, which I tested using a series of freely available LLMs. After experimenting with several that didn't really work, I ended up trying Anthropic's Claude, which proved far more effective than any of the others.

So this week I asked Claude to analyze those posts & to tell me how I'd done.

P.S. My H-GEAR project has its own early American LLM, by the way. It's been trained on a corpus of about 5,000 early American texts...

P.S. My H-GEAR project has its own early American LLM, by the way. It's been trained on a corpus of about 5,000 early American texts from EvansTCP. It's another thought experiment and was built by my colleague Michiel van der Ree from the Data Science Team at the University of Groningen. Give me a holler if you're curious about it. It's pretty rad: https://eahc.mhvdr.nl/

So is H-GEAR: https://www.esciencecenter.nl/news/american-revolution/

Oh, and if you're interested in using LLMs as a thinking and research tool, give Claude a try. I was impressed. (More on this theme...

Oh, and if you're interested in using LLMs as a thinking and research tool, give Claude a try. I was impressed. (More on this theme later.)

So keep an eye on those closely contested swing states in the 13: Pennsylvania, North Carolina, and Georgia. They may be the keys to the...

So keep an eye on those closely contested swing states in the 13: Pennsylvania, North Carolina, and Georgia. They may be the keys to the whole thing.

But if the race is close in the 13 States and especially if she either loses the popular vote there or merely wins by a small margin, then...

But if the race is close in the 13 States and especially if she either loses the popular vote there or merely wins by a small margin, then her chances for winning the country as a whole may not be too promising.

So if we're going to continue the thought experiment and make a prediction for 2024, what might it be? Well, if the 13 States go for...

So if we're going to continue the thought experiment and make a prediction for 2024, what might it be? Well, if the 13 States go for Harris in significant numbers and not just by a slim majority, then she might be expected to win big elsewhere in the country, too, and so could be on track to win the national election in the popular vote as well as in the Electoral College.

For the whole of the twentieth century, there is a near-perfect match between the 13-States winner (EC & popular vote), with the...

For the whole of the twentieth century, there is a near-perfect match between the 13-States winner (EC & popular vote), with the exception of 1968, where Humphrey won the majority of the 13 States' EC votes and the plurality of their popular vote but lost to Nixon in the overall for both. But George Wallace was a significant third-party candidate that year & won Georgia, which threw things off enough to provide a different outcome.

2004 is a notable exception in this regard. Then, the 13-States winner in terms of the EC and the popular vote (Kerry) lost both the...

2004 is a notable exception in this regard. Then, the 13-States winner in terms of the EC and the popular vote (Kerry) lost both the national EC and popular vote (to GW Bush). But the 13-States margin of victory in the popular vote was pretty small, and so was the national margin. So it was close, at least.

The 13-States Index (13-SI?) for the popular vote is especially closely correlated with the winner of the national popular vote. This is...

The 13-States Index (13-SI?) for the popular vote is especially closely correlated with the winner of the national popular vote. This is true even when the Electoral College awards the presidency to the loser of the popular vote, as notoriously happened in 2000 and 2016 and many fear might happen again in 2024. The 13 States winners in both of those elections (in the Electoral College as well as the popular vote) also won the national popular vote.