{"id":46652,"date":"2021-06-24T00:00:00","date_gmt":"2021-06-24T07:00:00","guid":{"rendered":"https:\/\/griddb-linux-hte8hndjf8cka8ht.westus-01.azurewebsites.net\/blog\/time-series-analysis-with-griddb-and-python\/"},"modified":"2025-11-13T12:55:25","modified_gmt":"2025-11-13T20:55:25","slug":"time-series-analysis-with-griddb-and-python","status":"publish","type":"post","link":"https:\/\/www.griddb.net\/en\/blog\/time-series-analysis-with-griddb-and-python\/","title":{"rendered":"Time Series Analysis with GridDB and Python"},"content":{"rendered":"

In this tutorial, we will see how to analyze time-series data stored in GridDB using Python. The outline of the tutorial is as follows –<\/p>\n

    \n
  1. Loading the dataset using SQL and Pandas<\/li>\n
  2. Preprocess the data to deal with null, missing values, etc. <\/li>\n
  3. Build a classifier for our data<\/li>\n<\/ol>\n

    Prerequisites<\/h2>\n

    This tutorial assumes prior installation of GridDB, Python3, and the associated libraries. If you have not installed any of the below packages, go ahead and do it before continuing with the tutorial. 1. GridDB<\/a> 2. Python 3<\/a> 3. GridDB Python Client<\/a> 4. NumPy<\/a> 5. Pandas<\/a> 6. Matplotlib<\/a> 7. Scikit-learn<\/a> 8. Lightgbm<\/a> 9. Seaborn<\/a><\/p>\n

    The following tutorial is carried out in Jupyter notebooks (Anaconda Navigator)<\/a>. You can install these packages directly in your environment using conda install package_name<\/code>. Alternatively, type pip install package_name<\/code> in the command prompt\/terminal.<\/p>\n

    Importing the necessary libraries<\/h2>\n

    Once you are done installing the required packages, let’s import these libraries.<\/p>\n

    \n
    import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom sklearn.preprocessing import *\nfrom sklearn.model_selection import *\nfrom sklearn.metrics import *\nimport os\nfrom datetime import datetime\nimport time\nfrom lightgbm import LGBMRegressor\nimport seaborn as sns\nfrom sklearn import metrics<\/code><\/pre>\n<\/div>\n
    \n
    APP_PATH = os.getcwd()\nAPP_PATH<\/code><\/pre>\n<\/div>\n
    'C:\\Users\\SHRIPRIYA\\Desktop\\AW Group\\GridDB'\n<\/code><\/pre>\n
    Loading the dataset<\/h2>\n
    The time-series dataset we’re using for this tutorial has been open-sourced on Kaggle<\/a>. The zip folder contains two separate files for training and testing. However, since the test dataset does not contain the labels, we will not be able to verify our model’s performance. Therefore, will be using the training file as the whole dataset, and later on, we will split it into Train and Test sets.<\/p>\n
    The training file has about 48000 rows (or instances) with 4 columns (or attributes) – ID, DateTime, Junction, and Vehicles<\/code>. Column Vehicle<\/code> is the dependent (or response) variable while DateTime and Junction<\/code> are independent (or explanatory) variables<\/p>\n
    Using SQL<\/h2>\n
    You can type the following statement in your Python script or console to retrieve the data from GridDB<\/a>. The advantage of using GridDB’s python-client<\/a> is that the resulting data type is a pandas dataframe. This makes data manipulation much easier.<\/p>\n
    statement = ('SELECT * FROM train_ml_iot')
    \ndataset = pd.read_sql_query(statement, cont)<\/code><\/p>\n
    The output will look like –<\/p>\n
    \"\"<\/a><\/p>\n
    Getting to know the dataset<\/h2>\n
    Now that we have loaded our dataset, it is time to get a peek at how it looks. We can print out the first 5 rows using the head<\/code> command. If you want to print more rows, simply pass a number to the function as an argument. For instance, dataset.head(15)<\/code> will print out the first 15 rows. You can also use the tail<\/code> command to get a gist of the dataset. The only difference is, as the name suggests, it prints the last 5 rows.<\/p>\n
    \n
    dataset.head()<\/code><\/pre>\n<\/div>\n
    \n