**************************************************************************** **************************************************************************** * Stata code to batch download and merge xSub data * this version: October 14, 2018 * * The following code replicates some of the basic functionality * of the xSub R package (https://CRAN.R-project.org/package=xSub) * for users of Stata. * * Questions, bugs, comments: xsub-project@umich.edu * **************************************************************************** **************************************************************************** * ************************************************************ * xSub file name format ************************************************************ * **** * Spatial panel data **** * * Format: xSub_SOURCE_COUNTRY_space_time * * SOURCE: short code of source * (see http://www.cross-sub.org/about/data/data-sources) * COUNTRY: ISO3 country code * (3 characters, e.g. AFG, IRQ; see http://www.iso.org/obp/ui) * space: spatial unit * (adm0=country, adm1=province, adm2=district, * priogrid=grid cell, clea=electoral constituency) * time: temporal unit * (year, month, week, day*) * *daily data only available for adm0, adm1 * **** * Event-level data **** * * Format: xSub_SOURCE_COUNTRY_event * * SOURCE: short code of source * (see http://www.cross-sub.org/about/data/data-sources) * COUNTRY: ISO3 country code * (3 characters, e.g. AFG, IRQ; see http://www.iso.org/obp/ui) * * * ************************************************************ * Set directory ************************************************************ // change directory to folder where you'd like to download the files cd "~/Downloads" ************************************************************ ************************************************************ * One-at-a-time file download examples ************************************************************ ************************************************************ * * Replace "xSub_GED_AFG_adm0_year" (no extension) * with name of file you'd like to download * ************************************************************ * Download single file, unzip, save as dta (spatial panel) ************************************************************ clear // download and save copy "http://cross-sub.org/api/data/xSub_GED_AFG_adm0_year.zip" "temp.zip", replace // unzip unzipfile "temp.zip", replace // open csv file import delimited "xSub_GED_AFG_adm0_year.csv", encoding(UTF-8) clear // save to disk save "xSub_GED_AFG_adm0_year.dta", replace // inspect downloaded data set more off, perm sum ************************************************************ * Download single file, unzip, save as dta (event level) ************************************************************ clear // download and save copy "http://cross-sub.org/api/data/xSub_GED_AFG_event.zip" "temp.zip", replace // unzip unzipfile "temp.zip", replace // open csv file import delimited "xSub_GED_AFG_event.csv", encoding(UTF-8) clear // fix invalid variable names rename lat latitude rename v8 longitude // save to disk save "xSub_GED_AFG_event.dta", replace // clean up erase "temp.zip" erase "xSub_GED_AFG_event.csv" // inspect downloaded data set more off, perm sum ************************************************************ ************************************************************ * Multiple file download & merge example ************************************************************ ************************************************************ * * Replace "xSub_GED_AFG_adm0_year" "xSub_GED_IRQ_adm0_year" * with names of files you'd like to download * ************************************************************ * Loop over file list (spatial panel) ************************************************************ clear foreach filename in "xSub_GED_AFG_adm0_year" "xSub_GED_IRQ_adm0_year" { // download and save copy `"http://cross-sub.org/api/data/`filename'.zip"' "`filename'.zip", replace // unzip unzipfile "`filename'.zip", replace // open csv file import delimited "`filename'.csv", encoding(utf8) clear // save to disk save "`filename'.dta", replace // clean up erase "`filename'.zip" erase "`filename'.csv" } // append clear append using "xSub_GED_IRQ_adm0_year.dta" "xSub_GED_AFG_adm0_year.dta", force // inspect set more off, perm sum ************************************************************ * Loop over file list (event level) ************************************************************ clear foreach filename in "xSub_GED_AFG_event" "xSub_GED_IRQ_event" { // download and save copy `"http://cross-sub.org/api/data/`filename'.zip"' "`filename'.zip", replace // unzip unzipfile "`filename'.zip", replace // open csv file import delimited "`filename'.csv", encoding(utf8) clear // fix invalid variable names rename lat latitude rename v8 longitude // save to disk save "`filename'.dta", replace // clean up erase "`filename'.zip" erase "`filename'.csv" } // append downloaded files clear append using "xSub_GED_IRQ_event.dta" "xSub_GED_AFG_event.dta", force // inspect merged data set more off, perm sum