I created a weather forecasting Web API with Laravel 8.83.11. The source code is available at the following github repository.
https://github.com/fukagai-takuya/weather-forecast
# The weather forecast data is obtained from other sites via Web API and stored in a database. This is a simple Web API program using Laravel.
# This blog page describes only an overview of the program and how to check its operation. The commands that were executed when the program was created and the contents of the program will be described in a separate blog page.
1. Summary of the program
1.1. When a query is received via the Web API specifying a date and time, it returns weather data for five cities: New York, London, Paris, Berlin, and Tokyo for that date and time. The time is not Japan Standard Time (JST) but Coordinated Universal Time (UTC), and the query is assumed to be made with the GET method. It is tested on a local server with Ubuntu 20.04.
When you make a Web API query like the following,
http://localhost:8000/api/get-weather-forecast?date=2022-05-13 10:25:49
If weather forecast data exists for the specified date and time, the following response will be returned. (It returns weather forecast data at 3-hour intervals. It selects a date and time within 1.5 hours before or after the specified date and time.)
{
"Result": "Success",
"id": "9",
"dt": "1652432400",
"dt_txt": "2022-05-13 09:00:00",
"new_york_main": "Clouds",
"new_york_description": "overcast clouds",
"london_main": "Clouds",
"london_description": "overcast clouds",
"paris_main": "Clouds",
"paris_description": "scattered clouds",
"berlin_main": "Clouds",
"berlin_description": "broken clouds",
"tokyo_main": "Rain",
"tokyo_description": "moderate rain",
"updated_at": "2022-05-12 07:56:09",
"created_at": "2022-05-12 06:11:20"
}
The Web API was tested using the testing tool Postman and the browser Chrome. The test with Chrome was performed by entering the following address in the URL input field.
http://localhost:8000/api/get-weather-forecast?date=2022-05-13 10:25:49
1.2. Weather forecast data are retrieved at 6-hour intervals by using the Web API listed on the 5 Day/3 Hour Forecast page at the following site. 3-hourly weather forecast data for the next 5 days can be retrieved. The retrieved data are stored in an SQLite database. Weather forecast data for stored date and time are updated with the new data.
https://openweathermap.org/api
1.3. When a user queries for weather forecast data using the Web API, this program first refers to the data stored in SQLite. If there is no weather forecast data for the relevant date and time, the program gets data from the external site. If the program still cannot obtain the weather forecast data, an error is returned. The following response is an example of an error response.
{
"Result": "Failed",
"Error": "No weather data was found for the specified date.",
"Date": "2022-06-12 10:25:40"
}
2. Preparing the environment for this program on Ubuntu 20.04
2.1. Get the source code from the repository with the following command.
$ git clone https://github.com/fukagai-takuya/weather-forecast
2.2. Change directory to weather-forecast directory.
$ cd weather-forecast
2.3. Prepare a Laravel environment configuration file (.env file).
Copy .env.weather-forecast with the following command and modify .env file as necessary.
$ cp .env.weather-forecast .env
2.4. Prepare the vendor directory with the following command. (It is assumed that composer is already installed.)
$ composer install
2.5. SQLite was used as the database. The following command will install SQLite and php’s driver for SQLite connections. (The second command is an example command for php 7.4.)
$ sudo apt install sqlite3 $ sudo apt-get install php7.4-sqlite3
Prepare an empty database file with the following command.
$ touch database/database.sqlite
Prepare a database table with the following command.
$ php artisan migrate
The following texts are output to the console and the database tables are created.
Migration table created successfully. Migrating: 2014_10_12_000000_create_users_table Migrated: 2014_10_12_000000_create_users_table (20.28ms) Migrating: 2014_10_12_100000_create_password_resets_table Migrated: 2014_10_12_100000_create_password_resets_table (24.93ms) Migrating: 2019_08_19_000000_create_failed_jobs_table Migrated: 2019_08_19_000000_create_failed_jobs_table (17.96ms) Migrating: 2019_12_14_000001_create_personal_access_tokens_table Migrated: 2019_12_14_000001_create_personal_access_tokens_table (25.99ms) Migrating: 2022_05_05_074622_create_jobs_table Migrated: 2022_05_05_074622_create_jobs_table (23.33ms) Migrating: 2022_05_06_013400_weather_data Migrated: 2022_05_06_013400_weather_data (34.54ms)
2.6. Crontab settings: The following configuration is designed to start the program every minute, but the program internally retrieves weather forecast data every 6 hours. The operation check described in this blog page can be done without crontab settings.
* * * * * cd /path-to-weather-forecast-project && php artisan schedule:run >> /dev/null 2>&1
3. Confirmation of operation on Ubuntu 20.04
3.1. Start the server for a quick operational check.
$ php artisan serve
3.2. Specify the date and time of the weather forecast data in the date parameter as shown below.
http://localhost:8000/api/get-weather-forecast?date=2022-05-13 10:25:49
A JSON-formatted response will be returned as described in 1.
Date and time is assumed to be in a format such as “2022-05-13 10:25:49”, so passing data in a different format such as “2022-05-14 10:25:4X” will result in the following response.
{
"Result": "Failed",
"Error": "Format Error",
"Date": "2022-05-14 10:25:4X"
}
Also, if you pass a date and time such as “2022-99-14 10:25:42”, you will get the following response.
{
"Result": "Failed",
"Error": "Incorrect Date",
"Date": "2022-99-14 10:25:42"
}
4. Test with PHPUnit
The following command is used to run a simple Feature test with PHPUnit.
$ php artisan test
If all tests pass, the following message is output to the console.
PASS Tests\Unit\ExampleTest ✓ example PASS Tests\Feature\ExampleTest ✓ example PASS Tests\Feature\WeatherForecastInquiryTest ✓ example ✓ get weather forecast ✓ get weather forecast empty input ✓ get weather forecast format error ✓ get weather forecast incorrect date ✓ get weather forecast data will not be found ✓ get weather forecast data will be found ✓ weather data table columns ✓ artisan shedule run command ✓ artisan migrate fresh command Tests: 12 passed Time: 2.96s
5. Task Scheduler Testing
5.1. Change job launch interval
As described in section 2.6., configure crontab to run the following command at one-minute interval.
Here, the following command is executed directly from the terminal for testing.
$ php artisan schedule:run
Modify app/Console/Kernel.php as follows to check the operation. With the following modification, a job will be added to the queue immediately after the “php artisan schedule:run” command is executed.
protected function schedule(Schedule $schedule)
{
// $schedule->job(new WeatherForecastInquiryJob)->everySixHours();
$schedule->job(new WeatherForecastInquiryJob)->everyMinute();
}
5.2. Check that the following commands are executed without problems.
$ php artisan schedule:run $ php artisan queue:work
“php artisan schedule:run” is a command called from crontab every minute.
“php artisan queue:work” is a command that retrieves jobs from a queue and executes them. When the job is executed, weather forecast data is retrieved from the external site and stored in the database.
If all is well, you can run “php artisan schedule:run” and you will get the following output.
[2022-05-13T08:13:06+00:00] Running scheduled command: App\Jobs\WeatherForecastInquiryJob
Running “php artisan queue:work” will output the following messages.
[2022-05-13 08:13:33][10] Processing: App\Jobs\WeatherForecastInquiryJob [2022-05-13 08:13:34][10] Processed: App\Jobs\WeatherForecastInquiryJob
Since “php artisan queue:work” will continue to run, press Ctrl-C to terminate it.
5.3. Running “php artisan queue:work” with Supervisor
When using on a Linux server, configure the Supervisor to call the “php artisan queue:work” command as described in this document.
The following is an example of the commands and a configuration file.
Install Supervisor with the following command.
$ sudo apt-get install supervisor
I have prepared /etc/supervisor/conf.d/laravel-worker.conf with the following way.
[program:laravel-worker] process_name=%(program_name)s_%(process_num)02d command=php /path-to-weather-forecast-project/artisan queue:work --sleep=3 --tries=3 --max-time=3600 autostart=true autorestart=true stopasgroup=true killasgroup=true user=fukagai numprocs=8 redirect_stderr=true stdout_logfile=/var/log/supervisor/laravel-weather-forecast-worker.log stopwaitsecs=3600
Start Supervisor with the following command.
$ sudo service supervisor start
I ran the “php artisan schedule:run” command three times in a row as follows.
$ php artisan schedule:run $ php artisan schedule:run $ php artisan schedule:run
The following log was output to /var/log/supervisor/laravel-weather-forecast-worker.log.
[2022-05-13 07:14:16][7] Processing: App\Jobs\WeatherForecastInquiryJob [2022-05-13 07:14:16][8] Processing: App\Jobs\WeatherForecastInquiryJob [2022-05-13 07:14:17][8] Processed: App\Jobs\WeatherForecastInquiryJob [2022-05-13 07:14:17][9] Processing: App\Jobs\WeatherForecastInquiryJob [2022-05-13 07:14:17][7] Processed: App\Jobs\WeatherForecastInquiryJob [2022-05-13 07:14:17][9] Processed: App\Jobs\WeatherForecastInquiryJob
If the Supervisor is running, enter the following command.
$ sudo service supervisor status
The following output is shown in the console.
supervisord is running
Supervisor can be stopped with the following command.
$ sudo service supervisor stop