Abstract

Ecuador’s agricultural sector plays a strategic role in the national economy; however, agricultural data remains fragmented across heterogeneous and isolated sources, limiting integrated analysis and evidence-based decision-making. This study proposes and implements a Big Data analytics framework based on the Medallion architecture and interactive dashboards to integrate, process, and visualize agricultural indicators from INEC, ESPAC, Ecuador Open Data, and FAOSTAT for the 2010–2024 period. The proposed framework adopts the Team Data Science Process (TDSP) methodology and structures workflows into Bronze, Silver, and Gold layers using Databricks for scalable data ingestion, transformation, and dimensional modeling. Interactive dashboards were developed in Tableau Public to support dynamic analysis of agricultural production, trade, producer prices, losses, and producer profiles. A comparative performance evaluation between Databricks Free Edition and Azure Databricks was conducted using SQL analytical workloads and dashboard interaction tests. Results showed that Azure Databricks reduced query execution times by up to 57%, especially in aggregation and join operations. Usability validation with 31 agricultural stakeholders reported high acceptance levels, including a 100% recommendation rate and a data trust score of 4.45/5. The findings demonstrate that scalable and low-cost Big Data technologies can effectively support agricultural digital transformation.

IPC Classification

Keywords