dataset_creation/description_of_script.sh.txt

#!/bin/bash
read -p "Enter package name: " name
echo "Package name is $name"
read -p "Enter path for package source files: " dir_path
#read -p "Enter start year: " year_s
#read -p "Enter end year: " year_e

year_s=2005
year_e=2018

mkdir $dir_path/$name
cd $dir_path/$name

for year_dir in `seq $year_s $year_e`
do
    mkdir $year_dir
    cd $year_dir
    month_list="03 07"
    for month_dir in $month_list
        do
            mkdir $month_dir 
            cd $month_dir
            day_list="15 28"
            for day_of_month in $day_list
                do
                day_mom=`curl -q "https://snapshot.debian.org/archive/debian/?year=$year_dir&month=$month_dir" | grep $year_dir$month_dir$day_of_month | awk -F'"' '{print $2}' | awk -F '/' '{print $1}'| tail -n 1`
                echo "deb [trusted=yes] http://snapshot.debian.org/archive/debian/$day_mom/ stable main" > sources_$day_mom.list
                echo "deb-src [trusted=yes] http://snapshot.debian.org/archive/debian/$day_mom/ stable main" >> sources_$day_mom.list
                echo "deb [trusted=yes] http://snapshot.debian.org/archive/debian/$day_mom/ stable/updates main" >> sources_$day_mom.list
                echo "deb-src [trusted=yes] http://snapshot.debian.org/archive/debian/$day_mom/ stable/updates main" >> sources_$day_mom.list  
                cp sources_$day_mom.list /etc/apt/sources.list -f                                                                              
                apt-get update                                                                                                                 
                apt-get source --download-only --allow-unauthenticated $name
                sleep 5
            done
            cd $dir_path/$name/$year_dir
    done
    cd $dir_path/$name
done
cd $dir_path/
chmod 777 $name -R




Description of script "script.sh"
------------
read -p "Enter package name: " name
echo "Package name is $name"
read -p "Enter path for package source files: " dir_path
#read -p "Enter start year: " year_s
#read -p "Enter end year: " year_e

year_s=2005
year_e=2018
------------
In this section i've will be entered name of package that we want to be downloaded and are declared years that will be parsed for that.


------------
mkdir $dir_path/$name
cd $dir_path/$name
------------
In this section we create directory and will chage work directory in that location.


------------
for year_dir in `seq $year_s $year_e`
do
------------
Here we start the loop (will be parsed all years that are in sequence between variables year_s and year_e)

------------
    mkdir $year_dir
    cd $year_dir
    month_list="03 07"
------------
here, script will create direcotry for every year that was parsed and we declare a string type variable for months that will be parsed (03 - march and 07 - july)


------------
    for month_dir in $month_list
        do
            mkdir $month_dir 
            cd $month_dir
            day_list="15 28"
------------
Here in second loop we parse months that are declared in variable month_list and will create directory for every month and work directory will be change in this path. We declare vlariable for days that will be parse in every months.


------------
            for day_of_month in $day_list
                do
                day_mom=`curl -q "https://snapshot.debian.org/archive/debian/?year=$year_dir&month=$month_dir" | grep $year_dir$month_dir$day_of_month | awk -F'"' '{print $2}' | awk -F '/' '{print $1}'| tail -n 1`
                echo "deb [trusted=yes] http://snapshot.debian.org/archive/debian/$day_mom/ stable main" > sources_$day_mom.list
                echo "deb-src [trusted=yes] http://snapshot.debian.org/archive/debian/$day_mom/ stable main" >> sources_$day_mom.list
                echo "deb [trusted=yes] http://snapshot.debian.org/archive/debian/$day_mom/ stable/updates main" >> sources_$day_mom.list
                echo "deb-src [trusted=yes] http://snapshot.debian.org/archive/debian/$day_mom/ stable/updates main" >> sources_$day_mom.list  
                cp sources_$day_mom.list /etc/apt/sources.list -f                                                                              
                apt-get update                                                                                                                 
                apt-get source --download-only --allow-unauthenticated $name
                sleep 5
            done
------------
In third loop we parse every day declared in day_list variable (15 and 28) and we declare variable day_mom wich actualy is that folder that will indicate location of sources files. For that we use grep, awk and tail commands on output of command curl -q url_name
After we find day_mom (day momment for a better remember) we generate a new sources.list file with new repository for all years, months and days declared, we copy this new file in /etc/apt/ and we repalce existed one (for this section this script must be run as root). After that we will update repositoryes (apt-get update)
and we download source of desired package.



------------
            done
            cd $dir_path/$name/$year_dir
    done
    cd $dir_path/$name
done
cd $dir_path/
chmod 777 $name -R
------------
In this section all loops will be closed and pwath will be changed in up with one directory
chmod 777 $name -R - this line will change rights for package directory name in full rights for all users.