Make Predictions

Create a Crawler

Set Your Algorithm Loose on a World of Live Data Streams

Peter Cotton

Join Our Virtual Office Hours

Jump-start your participation by joining us for one of our collaborator discussions, hosted by founder Peter Cotton, PhD. 

Tuesday 8:00-9:00pm EST
Friday 12:00-1:00pm EST 

 

GitHub

The microprediction python client includes a description of some classes,  notably MicroCrawler, that make it easy to create a self-navigating algorithm. This will wander from data stream to data stream and (hopefully) find examples where it excels.

Here's what you need to do:

  1. Subclass MicroCrawler
  2. Modify the crawler's sample method. 
  3. Run it. 

Simple eh? If you have created a good time-series prediction algorithm, or simply know of one, you can modify the crawler's sample method. The repository contains an extensive list of crawler examples, some of which have their own dedicated tutorials. 

Learn As You Go

See our blog article demystifying some of the live data streams. These are real-world challenges, not textbook examples, and some directly impact operations in real-time. There isn't a better preparation for real-world work - or a better way to impress your future employer. 

Here you can: 

  • Perfect the use of online algorithms
  • Find relevant exogenous data for your models
  • Sharpen your competence in maintaining and improving models

Ready to start?

Python Module 1   R Module 1

 

 

A World of Possibilities

Whether you are writing a paper, benchmarking a time-series algorithm, assessing an in-house model or chasing cash prizes, this site microprediction.com exists to help you navigate microprediction.org - where the algorithms battle it out.

Knowledge Center

See the knowledge center for short video tutorials explaining how to run the default algorithm (which crawls from data stream to data stream) and then improve it. 

No Registration

Here you can remain anonymous forever if that's what you wish. Visit private keys for an explanation. 

 

Help Others

Use the set_repository method on your crawler to establish a link on the leaderboard back to your code. 

Win Prizes

Microprediction is about democratized community prediction. But prizes don't hurt. See the competitions listing, and good luck! 

The Winner Is... card with colorful background with defocused lights

Join The Elite

Congratulations to Excyst Goose, the most recent recipient of a $1,000 prize. Past winners have received interviews at top-tier hedge funds and fintech companies. 

Deploying live algorithms that make distributional predictions and gather data will help you stand out from candidates who make offline point estimates using tabular data prepared by someone else. Remember, you're not in kaggle anymore. 
How do I Start? card with a urban background

Tutorials

Some may prefer to dive into the README, but we've prepared step-by-step video tutorials here as well.   

Python Module 1

You can also use R, with more support coming soon. 

R Module 1

 
Mechanic lying and working under car at the repair garage

Diagnose Your Models

You can also predict data streams that you have created yourself. This is quite a useful pattern, as you can:

  • Benchmark your efforts
  • Use your competitors to alert you to shifts in regimes or model drift.
  • Use z-streams for outlier or anomaly detection.
  • Find better ways to pre-process or post-process data. 
See our blog article on interpreting emotions during the Presidential debates - quick work because it takes advantage of the fact that all data streams are automatically standardized. 

 

Frequently Asked Questions

See frequently asked questions and also the knowledge center.

Follow the Movement