Using Celery with Flask

In this post, I will talk about using Celery with Flask. I’m going to assume that you have completed this tutorial on Celery. Basic Flask app structure Assume that we have written a nicely working Flask app which now needs to do some backend processing. Here’s our basic Flask app structure: +–myapp_dir/ +–app/ +–static/ +–templates/ +–utils/ +–views/…

Storing Results in Celery is a Bad Idea

For months I had sleepless nights over Celery backend system crashes. I was using RabbitMQ initially and then switched to Redis and it didn’t change anything. The Problem My app would generate hundreds of data every hour after processing and RabbitMQ would run out of file descriptors to handle it and crash. With Redis, the…

Using Fabric With Flask

I recently came across Fabric, a library that can be used for deployment or system administration tasks. To check out the most basic usage refer this tutorial. I’m going to assume that now you have a basic understanding of Fabric. You run fabric using the fab command on the terminal. This is efficient if you are using the terminal…

Sandy Had Some Brandy

A testament to Sandy (our guitar player). Some of it written in Joycean style. Sandy had some brandy Mixed with black coffee Then he played a solo On his humbucker Fendor Strat the blues note Or maybe the fourth Pick the humming buck her Tuck her “Red wine” pearl Might be on a Marshall Or…

It Rained Today

It rained today The weather report had predicted before But some event would unfold I had never known It rained today And suddenly I felt that cold It had memories of the past I could feel it within me once more While I was still breathing My thoughts whirled inside If I cried, I wouldn’t…

Heroku Environment Settings with Flask

I have to write this post now lest I forget what I have to say. I have been developing a website for a dear friend Patrick Joseph. While I am almost done with the most bits of the puzzle, I have two really cool things to share about Heroku. SparkPost plugin (I will be covering this in…

Retries using td-agent

At my current workplace, we use td-agent for data collection purposes. As with any system, network endpoints can break when there are issues on the server. Let’s take the following data flow – Considerations td-agent fetches data from service A and then sends an HTTP API call to service B. service A is highly reliable…

Review: The Topper Prepares – Soum Paul

Soum mentioned about his book a few months back when I met him the second time in office. I never mustered up the courage to ask him anything about it. As I saw this book, I tried to pre-order and Flipkart hadn’t “released” the book even as Amazon kept selling off copies of the book….

Using td-agent to archive data from SQS

Oft times we need to archive data for billing or viewing traffic trends and so on. This data can come from either one or multiple application servers. This post describes the use of fluentd/td-agent as a data collector. td-agent is a data collector tool invented by Treasure Data. It is open sourced and maintained by…

UIKit Rocks

When I published this site in June, it was insane. I had spent an entire day trying to rewrite the whole UI on UIKit and it didn’t work. With a splitting headache and spending 12 hours in front of my laptop I finally decided, to surrender to Bootstrap. After almost losing hope with UIKit and…

Time and Travel

My name is Idris Maymoon Samataar Guleed. Previously of Dikhil. Now an eternal traveller seeking the measure of man and earth! An excerpt from Idris by Anita Nair As I read this the first time, it intrigued me. How did Anita Nair manage to capture such a phenomenally simple idea? Read it again, you’ll find…

Homeless

My skin withers My legs grow colder And I sing for joy The night slowly darkens I sit in the pavement As I watch the road Moving cars in fast lanes Kids watching my suitcase My home is gone And when the wind blows I shudder and feel old Drunk my sorrows, no regrets Tonight…