Real Time Price Discovery on e-Marketplace

REAL TIME PRICE DISCOVERY AT e-MARKETPLACE

Every crop season, State Governments authorize a district-level technical committee to determine the district-wise, crop-wise input cost per hectare. These costs include seed, labour, fertilizer, pesticide, and irrigation costs. This input cost is called the Scale of Finance (SoF). It is the amount that is advanced as a loan to the farmer by various lending agencies. The amount varies by crop and district.

Now, artificial intelligence is being applied to satellite imagery to arrive at the average output from the farm in a district (RMSI is carrying out this exercise for the Government of India, including three districts of Telangana). For illustration purpose, this yield can be multiplied by the minimum support price (the price at which the government buys produce) to estimate a farmer’s revenue.

In 2017, the average actual soybean yield in Sheopur, Madhya Pradesh, was 11.9 quintals per hectare (official government figure). Multiply this with the minimum support price (MSP) of Rs 3,050 per quintal to get an estimate of the farmer’s average revenue per hectare of Rs 36,295. That year, the government had declared Rs 34,340 per hectare as the input cost for soybean in Sheopur. Therefore, by the government’s estimate, an average farmer in this district made a margin of less than Rs 2,000 per hectare for a crop season of 6 months. With average farm-holding at less than 2 hectares, the government numbers showcase that most farmers in Sheopur did not make a margin of more than Rs 3,600 in a crop season of 6 months.

Sheopur was a relatively happier district in 2017. Not too far away, in Indore, the average actual soybean yield was 11.5 quintals per hectare. Still, because the input cost in Indore was higher at Rs 40,000 per hectare, by the government’s admission (yield, MSP, and input cost – all government numbers), the average soybean farmer in Indore lost Rs 5,000 per hectare.

Summary Snapshot from the data

*Scale of Finance & Minimum Support Price data has been sourced from the publically available databases.

Through this initiative, the Government of Telangana is demonstrating its intent to bring the crop prices to economically viable levels and ensure that the farmers can realize the same. It is well known that most farmers in India sell below MSP. A few of the most prominent reasons for the same are:

• There is no mechanism through which farmers can set the floor price.
• Farmers come to the mandis from villages for a day or two to sell their produce, and don’t want to incur the cost of elongating their stay in the hope of getting a better price.
• Since MSP purchases start late in the season, it’s doesn’t directly benefit the farmer. There is no methodology currently in place to stagger/ plan the mandi arrivals of the crops.

So, it becomes imperative for all the stakeholders to adopt the e-marketplace practices for real-time price discovery and fair and transparent business transactions in a secured way.

APPROACH TO PRICE DISCOVERY AT e-MARKETPLACE
RMSI has used the following approach to develop the solution for this challenge:

1- Understanding the input data – The following datasets yield data, market yard prices data, weather data, market yard locations data and warehouse data are provided on the website for model development. The complete datasets have been analyzed and filtered; few fields like arrivals, minimum price, maximum price and total yield have high correlations from modeled data, i.e., target column (with more detailed data, a model can be developed using other variables as well).

2- Prepare the training data – Based on the understanding of input data, we have selected the columns from different datasets and created a separate input source for model development. The data was filtered based on the crop and AMC. The data was further divided into train and test data set in the ratio of 70:30. Input samples are given below:

Input Columns – Arrivals, Minimum, Maximum and Yield
Target Columns – Actual/Model

3- Data cleaning, normalization and scaling – After analyzing datasets, we found that the columns like Arrivals, Maximum, Minimum and Model values are 0 in number of rows, so, this data was removed. There are around 42k data points available in market yard prices data sets. After cleaning the data sets, around 19K data sets were left for model development.

The next steps were to detect the outliers, where a manual approach was applied and data with huge variations was removed. Post which, data normalization was implemented using the standard machine learning techniques. We applied the standard scaling in this approach.

4- Model development and fine-tuning – We tried various algorithms and techniques of regressions due to lack of seasonality in input data; we did not apply the time series analysis. We applied Random forest, decision tree, linear and SVM regression. The linear regression algorithms gave best performance in our case. We optimized all the hyper parameters for best result.

5- Develop the accuracy matrices and compare the model’s performance – The accuracy matrices R2 was used to validate the model performance.
-----------------------------------------------------------------------------------
Please refer to the detailed Solution Approach Note (with images), Solution Output and Presentation in the zipped folder for more information.