{"id":46736,"date":"2022-11-30T00:00:00","date_gmt":"2022-11-30T08:00:00","guid":{"rendered":"https:\/\/griddb-linux-hte8hndjf8cka8ht.westus-01.azurewebsites.net\/blog\/studying-and-forecasting-web-traffic-using-python-and-griddb\/"},"modified":"2025-11-13T12:56:21","modified_gmt":"2025-11-13T20:56:21","slug":"studying-and-forecasting-web-traffic-using-python-and-griddb","status":"publish","type":"post","link":"https:\/\/www.griddb.net\/en\/blog\/studying-and-forecasting-web-traffic-using-python-and-griddb\/","title":{"rendered":"Studying and Forecasting Web Traffic using Python and GridDB"},"content":{"rendered":"

In the last few years, the time-series database category has experienced the fastest growth. Both established and emerging technology sectors have been producing an increasing amount of time-series data.<\/p>\n

The quantity of sessions in a given period of time is known as web traffic, and it varies greatly depending on the time of day, day of the week, and other factors. The amount of web traffic a platform can handle is determined by the size of the servers that host the platform.<\/p>\n

Based on historical visitor volume data or historical web traffic data, you can dynamically allocate many servers. And that brings us to the data science challenge, which is basically analysing and forecasting the volume of sessions or web traffic based on past data.<\/p>\n

The outline of the tutorial is as follows:<\/p>\n

    \n
  1. Dataset overview<\/li>\n
  2. Importing required libraries<\/li>\n
  3. Loading the dataset<\/li>\n
  4. Analysing with data visualization<\/li>\n
  5. Forecasting<\/li>\n
  6. Conclusion<\/li>\n<\/ol>\n

    Prerequisites and Environment setup<\/h2>\n

    The full jupyter file and be found in our GitHub repo:<\/p>\n

    $ git clone –branch web-forecasting https:\/\/github.com\/griddbnet\/Blogs.git<\/p>\n

    This tutorial is carried out in Anaconda Navigator (Python version \u2013 3.8.5) on Windows Operating System. The following packages need to be installed before you continue with the tutorial \u2013<\/p>\n

      \n
    1. \n

      Pandas<\/p>\n<\/li>\n

    2. \n

      NumPy<\/p>\n<\/li>\n

    3. \n

      re<\/p>\n<\/li>\n

    4. \n

      Matplotlib<\/p>\n<\/li>\n

    5. \n

      Seaborn<\/p>\n<\/li>\n

    6. \n

      griddb_python<\/p>\n<\/li>\n

    7. \n

      fbprophet<\/p>\n<\/li>\n<\/ol>\n

      You can install these packages in Conda\u2019s virtual environment using conda install package-name<\/code>. In case you are using Python directly via terminal\/command prompt, pip install package-name<\/code> will do the work.<\/p>\n

      GridDB Installation<\/h3>\n

      While loading the dataset, this tutorial will cover two methods \u2013 Using GridDB as well as Using Pandas. To access GridDB using Python, the following packages also need to be installed beforehand:<\/p>\n

        \n
      1. GridDB C-client<\/a><\/li>\n
      2. SWIG (Simplified Wrapper and Interface Generator)<\/li>\n
      3. GridDB Python Client<\/a><\/li>\n<\/ol>\n

        1. Dataset Overview<\/h2>\n

        The dataset consists of approximately 145k time series. Each of these time series represent a number of daily views of a different Wikipedia article, starting from July, 1st, 2015 up until December 31st, 2016.<\/p>\n

        https:\/\/www.kaggle.com\/competitions\/web-traffic-time-series-forecasting\/data<\/p>\n

        2. Importing Required Libraries<\/h2>\n
        \n
        %matplotlib inline\nimport pandas as pd\nimport numpy as np\nimport re\nimport seaborn as sns\nfrom fbprophet import Prophet\nimport griddb_python as griddb\n\nimport matplotlib.pyplot as plt\nplt.style.use('fivethirtyeight')\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")<\/code><\/pre>\n<\/div>\n
        3. Loading the Dataset<\/h2>\n
        You can download the dataset for yourself here: https:\/\/www.kaggle.com\/competitions\/web-traffic-time-series-forecasting\/data?select=train_1.csv.zip<\/p>\n
        Let\u2019s proceed and load the dataset into our notebook.<\/p>\n
        3.a Using GridDB<\/h3>\n
        Toshiba GridDB\u2122 is a highly scalable NoSQL database best suited for IoT and Big Data. The foundation of GridDB\u2019s principles is based upon offering a versatile data store that is optimized for IoT, provides high scalability, tuned for high performance, and ensures high reliability.<\/p>\n
        To store large amounts of data, a CSV file can be cumbersome. GridDB serves as a perfect alternative as it in open-source and a highly scalable database. GridDB is a scalable, in-memory, No SQL database which makes it easier for you to store large amounts of data. If you are new to GridDB, a tutorial on reading and writing to GridDB<\/a> can be useful.<\/p>\n
        Assuming that you have already set up your database, we will now write the SQL query in python to load our dataset.<\/p>\n
        The read_sql_query function offered by the pandas library converts the data fetched into a panda data frame to make it easy for the user to work.<\/p>\n
        \n
        sql_statement = ('SELECT * FROM train_1.csv)\ndf1 = pd.read_sql_query(sql_statement, cont)<\/code><\/pre>\n<\/div>\n
        Note that the cont<\/code> variable has the container information where our data is stored. Replace the credit_card_dataset<\/code> with the name of your container. More info can be found in this tutorial reading and writing to GridDB<\/a>.<\/p>\n
        When it comes to IoT and Big Data use cases, GridDB clearly stands out among other databases in the Relational and NoSQL space. Overall, GridDB offers multiple reliability features for mission-critical applications that require high availability and data retention.<\/p>\n
        3.b Using pandas read_csv<\/h3>\n
        We can also use Pandas’ read_csv<\/code> function to load our data. Both of the above methods will lead to the same output as the data is loaded in the form of a pandas dataframe using either of the methods.<\/p>\n
        \n
        df = pd.read_csv('train_1.csv', parse_dates=True) <\/code><\/pre>\n<\/div>\n
        \n
        df.head()<\/code><\/pre>\n<\/div>\n
        \n