Council on Energy, Environment and Water Integrated | International | Independent
At Council on Energy, Environment and Water (CEEW), one of Asia’s leading not-for-profit research organisations, we focus on impacting sustainable development at scale using data, integrated analysis, and strategic outreach.
Data is an integral and indispensable component of our work, informing our research outputs across multiple domains. With a team of over 200 members from diverse backgrounds including law, political science, and engineering—we prioritize constant upskilling to guarantee efficient data collection, cleaning, and analysis.
Given the teams' diversity, we needed an organization with in-depth upskilling opportunities for our data professionals, while also providing beginner-friendly courses for newbies. After some research, we came across the DataCamp Donates program, which provides free state-of-the-art courses for non-profit research organizations. Our partnership with DataCamp has been a grand success for our Sustainable Food System team.
The Sustainable Food Systems team at CEEW is undertaking a five-year longitudinal assessment study to establish evidence on natural farming in the state of Andhra Pradesh, India. The study will follow more than 4,000 farmers for five years to capture and track their soil health and fertility, cost of cultivation, crop yield, income from the sale of agricultural produce, nutritional output from farms, women’s dietary diversity, and women's empowerment.
Given the complexity of the questionnaire—which necessitates six different sections just to capture the costs of three classes of synthetic agrochemicals and bio-inputs—the 40-minute long baseline survey churned out a dataset with more than 7,000 variables. With at least four surveys per year, the data pertaining to revenue and expenditure for each farmer have to be combined from all the datasets to obtain insights on annual farm revenue and profit.
Besides the survey, the team also had to process large secondary datasets on various agriculture-related components, which fed into the sampling methodology of the primary survey. For context, the team had to analyze crop-wise cultivated areas at the sub-district level for 154 crops. This data was stored in 154 different sheets (one sheet for each crop) in a single Excel file, and the team had to process 39 such files quickly.
Given the enormity of the task, the team had to upskill themselves to clean, understand, and analyze these datasets in a short time. With either R or Python as options, the team weighed up the pros and cons and decided to go with Python, which can be used for a broad range of tasks from data visualization to web scraping.
The team enrolled in DataCamp Python courses and tracks played a vital role in their upskilling. From Introduction to Python to Data Manipulation with pandas, the team built their expertise rapidly. They could achieve this through three features of DataCamp’s courses.
The expertise built through DataCamp equipped the team to upload and resize the secondary data into a single dataset for analysis within days through Python. Throughout, the team extensively flagged and report outliers in various sections, including farm expenditure, yield, and revenue, while also analyzing the data to understand the agronomic conditions.
Building on this foundation, the SFS team aims to leverage the DataCamp Donates program even further, with a goal to perform statistical analysis (including performing balance tests, winsorizing data, applying sampling weights, etc.) in Python. The team has also selected relevant courses on statistics, sampling, etc. to achieve this.
“With basic introductory knowledge in Python, I upskilled myself in Python through DataCamp to clean and analyze large datasets within three weeks. The interactive exercises were helpful in learning data analysis on the go. I am able to handle data in Python as easily as I could in MS Excel now."
– Satheskumar, CEEW, Research Analyst
“Among several programming and data science learning platforms that I have tried over the years, Data Camp is certainly the most intuitive and effective. Their support to provide free access to their platform for a non-profit organization like ours is highly commendable. It has helped many in our team to take a deeper plunge in their data analysis journey to shape evidence-backed policies.”
– Abhishek Jain, CEEW, Fellow & Director - Powering Livelihoods
In conclusion, the partnership between CEEW and DataCamp Donates program has had a significant impact on the Sustainable Food Systems team's ability to handle large datasets and perform data analysis. The team's upskilling through DataCamp has enabled them to collect, clean, and analyze data efficiently, which is crucial for their five-year longitudinal assessment study on natural farming in India. DataCamp's interactive and hands-on exercises have played a vital role in building the team's expertise rapidly. Overall, this partnership highlights the value of upskilling programs and data literacy for non-profit research organizations' sustainable development efforts.