Beyond DeSci Part 3: From Concept to Code
A working prototype tackles the challenges of scientific validation online
TL;DR: The SciValidate concept has evolved from theoretical discussion to working code with a functional prototype available at scivalidate.org/example. Many technical and social challenges remain. Join the collaborative development effort via the GitHub repository to help build a better system for scientific trust online.
After exploring the SciValidate concept in the two previous posts, I'm excited to share concrete progress. What began as a theoretical discussion about modernizing scientific trust has evolved into working code, including a functional prototype you can explore at scivalidate.org/example. I frequently contemplated turning this into a startup, but ultimately, I believe it is better off as a community-based open-source project.
I'll admit that Claude AI deserves substantial credit for translating my English sentences into Python, JavaScript, SQL, and HTML—though not without errors and misunderstandings that required significant correction. The documentation was similarly AI-assisted but human-verified. As with all AI-adjacent projects, vigilance against hallucinations and misinterpretations remains essential. I continue to monitor and correct the GitHub repository as needed.
From "Likes" to Scientific Validation: A Working Prototype
The core of SciValidate is a visual system replacing binary social media engagement metrics ("likes") with meaningful indicators of scientific validity. Our first challenge was establishing an author’s reputation—a concept lacking in current online discourse.
To address this, I've populated a small database with public data from one academic department, inferring reputation scores from two key public assets: ORCID for identity tracking and OpenAlex for keywords and coauthor networks. This groundwork enables the development of a computational model representing expertise networks that, once fully implemented, will allow engaged communities to evaluate purported facts objectively.
The current prototype offers a view "under the hood" where faculty members and their publications have been analyzed by keyword to determine plausible "reputation scores" (0-10) in particular scientific fields. While the scoring system offers flexibility in weighing different factors, it also introduces challenges. If I know anything about scientists, they're competitive—keeping score is simultaneously this approach's beauty and peril.
Eventually, the system will extend these badges to communicate scientific consensus through distinctive visual indicators (as described in my previous installment). These provide immediate context that "likes" and shares cannot, helping readers distinguish between mere popularity and scientific validation. The interactive badges will offer access to detailed validation interfaces showing evidence distribution, expert perspectives, and research context.
In short, just because one clown with an advanced degree says something doesn't make it Science.
The Scale Challenge: Building the Seed Network
Even with substantial resources, several significant challenges remain. The first among these is scalability.
For the prototype, I selected Rensselaer Polytechnic Institute's Chemistry Department as a test case (a choice driven by personal history). Of their 32 faculty members, I identified 14 with unambiguous ORCID identifiers and substantial publication records in OpenAlex. Analysis of their publication networks revealed connections to over 5,000 coauthors across multiple institutions and disciplines, resulting in a database exceeding 10 MB.
This highlights our first scaling challenge: starting with just one department, it quickly expanded to thousands of potential participants. The technical limitations are substantial—OpenAlex permits retrieving only 100 publications per request, making data collection slow and redundant. Effectively mapping these relationships while maintaining data accuracy requires resources beyond a solo effort and "free" APIs.
At this stage, we must create a "forest of expertise" identifying key experts across different academic domains. These aren't just credentialed individuals but those whose judgment is generally respected by peers in their fields. We can anchor reputation scores based on existing community metrics and build toward an "earned" reputation for honesty and transparency that transcends conventional metrics. Then, we need user-friendly online tools for strategic platform rollout, adding value that encourages profile claiming and following.
It's crucial to emphasize that SciValidate isn't intended to elevate academic credentials as the sole measure of authority. I believe skilled science communicators should receive significantly higher ratings than deeply specialized academics who only communicate with narrow audiences. Our current academic-based approach is simply a starting point—a seed database of verifiable information while building toward a more comprehensive system valuing both expertise and effective communication.
The Integration Dilemma: Platform Independence
For SciValidate to achieve its purpose, the verification system must seamlessly integrate with existing communication platforms. Validated users should be able to include verification badges in posts across Twitter/X, LinkedIn, Substack, and other platforms where scientific discourse occurs, including the scientific literature.
Unfortunately, most major platforms are proprietary and closed, with limited opportunities for third-party integration. Some preliminary exploration revealed significant hurdles:
API limitations: Most platforms restrict what third-party applications can do, particularly regarding content modification or embedding.
Divergent standards: Each platform has different technical requirements and user interfaces, requiring multiple implementation approaches.
Platform resistance: Social media companies have business models that monetize engagement. I would engage more with them if I thought they were a source of truth, but I’m not sure everyone would.
One potential approach is focusing initially on platforms within the Fediverse (like Mastodon or BlueSky), which typically offer more open integration possibilities. However, this would significantly limit reach, as most scientific communication still happens on mainstream platforms.
The Network Effect Barrier
The prototype demonstrates how SciValidate could work, but achieving widespread adoption faces the classic chicken-and-egg problem of networked platforms:
Visibility threshold: Until enough prominent scientists adopt the system, it lacks credibility and appeal to others.
Value demonstration: The benefits of participation aren't immediately apparent without a critical mass of users.
Institutional buy-in: Academic departments and research institutions need to see value before encouraging adoption among their researchers.
The "Facebook strategy" of starting with prominent institutions could work, but it requires significant marketing and outreach resources. Additionally, scientists are typically pragmatic adopters of new technologies, embracing tools only when they demonstrably solve immediate problems. Many have not yet embraced the new media.
Our initial implementation with RPI's Chemistry Department might provide the seed, but expanding beyond that will require deliberate community-building strategies and possibly strategic partnerships with scientific societies, journals, or funding agencies that could incentivize participation.
The Reputation Paradox
Perhaps SciValidate's most philosophically challenging aspect is defining and measuring scientific reputation. Any scoring system inevitably creates incentives that can be gamed or optimized to undermine its purpose.
Several competing considerations complicate this challenge:
Citation metrics vs. quality: Traditional measures like h-index reward publication volume and citation counts, which don't necessarily reflect research quality or integrity.
Expertise vs. authority: How do we acknowledge deep domain expertise without creating rigid hierarchies that stifle innovation?
Consensual vs. dissenting voices: Scientific progress often comes from minority viewpoints that challenge consensus. How do we value productive disagreement?
Cross-disciplinary contributions: Many significant advances happen at disciplinary intersections, where traditional reputation measures may not apply.
The algorithm currently infers reputation based on publicly available information rather than tracking individual behavior. This approach allows a fast start but is not the endgame. The system must evolve to recognize those who excel at verification, communication, and constructive debate—not just traditional academic metrics.
A Call for Collaborative Development
To address these challenges collaboratively, I've created a GitHub repository containing all the prototype code developed so far:
For those unfamiliar with GitHub, it's the world's leading platform for collaborative software development—think of it as a social network for code where developers can work together on projects. The repository includes:
Data Collection Tools: Scripts for gathering faculty information and publication data
Database Components: Systems for organizing researcher profiles, publication records, and expertise metrics
Analysis Modules: Algorithms for calculating reputation scores and building collaboration networks
Web Interface: The working verification badge system and interactive validation interface
Documentation: Detailed explanations of the technical challenges and architecture
The purpose of making this code publicly available is twofold:
Transparency: Demonstrating that SciValidate isn't just a theoretical concept but has functioning components that can be tested and improved
Collaboration: Inviting developers, researchers, and other technical specialists to contribute their expertise to solving these challenges
Toward Solutions: How You Can Help
If you share my vision for transforming how scientific expertise is verified online, here's how you can contribute:
For Technical Contributors
If you have skills in programming, data science, UX design, or system architecture, consider:
Exploring the GitHub repository: Examine the code, documentation, and current implementation
Contributing improvements: Fork the repository, make changes, and submit pull requests
Opening issues: Identify specific problems that need addressing
Joining discussions: Participate in technical conversations about implementation approaches
Even if you're not ready to write code, your insights on technical approaches could be invaluable.
For Scientific Contributors
If you're a researcher or academic:
Test the prototype: Provide feedback on the accuracy of expertise mapping
Suggest field taxonomies: Help refine the classification of scientific domains
Evaluate reputation metrics: Offer perspective on how scientific expertise should be measured
Identify use cases: Share how verification would benefit your field
For All Readers
Even without technical or scientific expertise, you can contribute by:
Sharing this initiative: Help us reach potential collaborators
Providing user feedback: Tell us what would make verification badges helpful to you
Suggesting platforms: Identify where scientific validation would be most valuable
Following progress: Sign up for updates at scivalidate.org/signup
Next Steps
To move this initiative forward, I'll be focusing on these concrete actions:
Enhancing the prototype: Adding more validation states and interaction models
Improving field classification: Refining the expertise mapping algorithms
Building open APIs: Creating interfaces for platform integration
Outreach to institutional partners: Seeking broader academic participation
The problems of scientific trust and verification in online spaces are too essential to abandon simply because the path forward is difficult. By sharing both the vision and the practical challenges as open source, I hope to attract collaborators who can help turn this concept into reality.
The GitHub repository represents a transition from a conceptual discussion to practical implementation, so my efforts will transition from this platform to that one. It won't solve all the challenges immediately, but it provides a foundation for collaborative problem-solving and incremental progress.
If you'd like to get involved, explore the GitHub repository at github.com/jburbs/scivalidate, try the prototype at scivalidate.org/example, or sign up for updates at scivalidate.org/signup.
Share your thoughts in the comments: Which of these challenges do you think is most critical to address first? Do you have expertise or connections to help overcome these hurdles?