# Should I Stream JSON or Poll JSON: A Practical Take

## **Streaming vs. Polling JSON: Choosing the Right Approach for Rubric Generation**

When designing the **rubric generation system**, I faced a key architectural decision:

> **Should I stream JSON responses as they are generated, or should I poll the backend for updates?**

Streaming sounds like the obvious choice for real-time applications, but after careful consideration, I chose **polling**. Here’s why.

## **Understanding the Problem: Rubric Generation is an Asynchronous Process**

Generating rubrics is not an instant operation. It involves multiple steps, each happening asynchronously:

1. **Analyze the uploaded question paper.**
    
2. **Extract questions and transform them into JSON.**
    
3. **Send each JSON block to OpenAI for evaluation (create tasks for OpenAI to generate rubrics for each question).**
    
4. **Store responses in the database for retrieval.**
    

Since each JSON response is generated independently, **streaming might seem like a good fit**. But does it actually make sense?

## **Challenges with Streaming JSON**

If I were to stream rubric data to the frontend, several key questions arose:

### **1) What if the user navigates away or kills the app?**

* **Streaming requires an active connection.** If the user leaves mid-process, they could **lose all streamed data.**
    
* Polling allows users to **resume tracking progress anytime**, even after navigating away.
    

### **2) When should we insert data into the database?**

* Should we **stream first** and insert later? If so, what happens if the database rejects some updates?
    
* Should we **insert first** and then stream? If so, how do we ensure updates are pushed properly?
    

### **3) What if one process breaks?**

* If **data insertion and streaming run separately**, one could fail while the other succeeds.
    
* This could lead to users seeing rubric updates **that don’t actually exist in the database**.
    
* To avoid this, I would need to **set up queues, event listeners, and retry mechanisms**, making the infrastructure much more complex.
    

At this point, **streaming JSON became a rabbit hole of complexity.** I realized that I would need to:  
1\. Maintain a **queue** to persist data while streaming.  
2\. Set up **event listeners** to detect failures and retry processing.  
3\. Ensure that streaming doesn’t **outpace database updates**.

This added **unnecessary complexity** at both the infrastructure and implementation levels.

## **Polling JSON: A More Predictable & Reliable Approach**

While polling is often seen as an "archaic" approach, it actually **provides predictability and consistency** in this case.

Here’s how the **rubric generation flow** works with polling:

1. **User uploads a question paper.**
    
2. **The system extracts questions and converts them into JSON.**
    
3. **A background job is started** to generate rubrics for the entire question set.
    
4. **As the LLM generates rubrics for each question, we insert them into the database** under a common **parent ID**.
    
5. **Once all questions have been processed, the job is marked as finished.**
    
6. **The frontend polls an API** that aggregates all rubrics linked to the **parent ID** and displays them in real time.
    

## **Why Polling Wins in This Scenario**

**More Reliable:** Polling always retrieves data from the database, ensuring consistency.  
**Handles User Navigation:** Users can leave and return at any time without losing progress.  
**Simpler Infrastructure:** No need for persistent connections, queues, or event-driven architecture.  
**Predictable Execution:** Each poll fetches exactly what’s in the database—nothing more, nothing less.

## **Final Thoughts: Occam’s Razor in Action**

Occam’s Razor states:  
*"The simplest solution that meets all requirements is usually the best."*

**Polling ensures real-time updates while avoiding the complexities of streaming, queue management, and event handling.**

For use cases like **chat apps or stock tickers**, **streaming makes sense**. But for a job-based system like rubric generation, **polling is the better choice.**  
  
A short video of rubrics polling in action.

%[https://www.youtube.com/watch?v=yPOsxMKzxVk]