{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predefined styles in Magics, ERA5 NetCDF data example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Magics comes with number of ways to define shading when visualizing your data.  \n",
    "If you want full control you can choose number of levels, the way levels are calculated, contours or no contours, shading or no shading, type of shading, showing grid values or not. Magics doesn't lack number of parameters you can set!\n",
    "\n",
    "Many times, we don't have time to make a style that fits partucular data, we just want to inspect data for further use. For this purpose Magics has built in predefined styles for large number of meteorological and climatological variables."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## First import Magics, cdsapi and the other libraries\n",
    "\n",
    "Aside of Magics, in this Notebook we will use couple of libraries to make work easier.  \n",
    "\n",
    "**cdsapi** to download data from Climate Data Store. All the data on CDS are free to use, but make sure you have an account and api key, and accept terms of service for the dataset you want to use, before the start.  \n",
    "Documentation about **cdsapi** can be find [here](https://cds.climate.copernicus.eu/api-how-to.com \"CDS API how to\")  \n",
    "\n",
    "**datetime** for work with date and time objects.  \n",
    "**ipywidgets** for Jupyter Notebook widgets.  \n",
    "**xarray** for extracting data and metadate from NetCDF file.  \n",
    "\n",
    "At ECMWF make sure that Magics and cdsapi are loaded.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import Magics.macro as magics\n",
    "import cdsapi\n",
    "import datetime\n",
    "import ipywidgets as widgets\n",
    "from ipywidgets import Layout, Button, Box, VBox\n",
    "import xarray as xr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`menus.py` contains the code got menus to choose parameter, type, date and time..  \n",
    "We're just gonna run it here so our nice widgets work, but the code is not important for this excersize."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "57004f232ee14e19b42b7d3b88df45b0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Box(children=(Dropdown(description='Choose parameter:', options=('divergence', 'fraction_of_clo…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%run -i 'netcdf_menu.py'\n",
    "\n",
    "display(VBox([box_auto]))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we have code for CDS API request.  \n",
    "It uses parameters we have chosen before, so we don't have to change the code and so we reduce the posibility of an error.  \n",
    "At the moment this request only works for pressure levels fields, and there is separate Notebook for surface fields. In future it might be merged in one Notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2018-10-11 09:08:54,783 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-pressure-levels\n",
      "2018-10-11 09:08:54,813 INFO Request is queued\n",
      "2018-10-11 09:08:55,821 INFO Request is running\n",
      "2018-10-11 09:08:59,588 INFO Request is completed\n",
      "2018-10-11 09:08:59,589 INFO Downloading http://136.156.132.110/cache-compute-0002/cache/data1/adaptor.mars.internal-1539245335.267453-25627-4-9615c028-bd22-4ecb-9bd8-c51dc6d0d77b.nc to download.nc (3.1M)\n",
      "2018-10-11 09:08:59,676 INFO Download rate 35.8M/s\n"
     ]
    }
   ],
   "source": [
    "c = cdsapi.Client()\n",
    "\n",
    "a = c.retrieve(\n",
    "    'reanalysis-era5-pressure-levels',\n",
    "    {\n",
    "        'variable':[\n",
    "            parameter.value\n",
    "        ],\n",
    "            'grid': ['0.2', '0.2'],\n",
    "            'product_type': Type.value,\n",
    "            'pressure_level' : level.value,\n",
    "            'year': dates.value.year,\n",
    "            'month': dates.value.month,\n",
    "            'day': dates.value.day,\n",
    "            'time': times.value,\n",
    "        'format':'netcdf'\n",
    "    },\n",
    "                \"download.nc\"\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load netcdf data and set area, coaslines and default tittle:\n",
    "\n",
    "When loading NetCDF file, we need to tell Magics exactly which variable we want to plot.  \n",
    "We might need the help of **xarray** to check Netcdf variable name in our file, and pass it to Magics.\n",
    "\n",
    "More about loading grib files in Magics can be found [here](https://confluence.ecmwf.int/display/MAGP/Netcdf+Input \"NetCDF input\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Variable to plot is:  r\n"
     ]
    }
   ],
   "source": [
    "ds = xr.open_mfdataset('download.nc')\n",
    "variable_name = list(ds.data_vars)[0]\n",
    "\n",
    "print('Variable to plot is: ', variable_name)\n",
    "data =  magics.mnetcdf(netcdf_filename  = 'download.nc',\n",
    "                      netcdf_value_variable = variable_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a4ed2a2be0e9484ca31722ad3fd062f5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Dropdown(description='Predefined areas:', index=15, options=('north_pole', 'eurasia', 'global', 'south_west_eu…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "areas = magics.predefined_areas()\n",
    "Area = widgets.Dropdown(options=areas, value = 'europe', description='Predefined areas:', style=style)\n",
    "display(Area)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "projection = magics.mmap(\n",
    "    subpage_map_library_area = \"on\",\n",
    "    subpage_map_area_name = Area.value\n",
    ")\n",
    "\n",
    "coast = magics.mcoast(map_grid =  \"on\",\n",
    "                      map_grid_colour  =  \"charcoal\",\n",
    "                      map_coastline  =  'on',\n",
    "                      map_coastline_colour =  'navy',\n",
    "                      map_boundaries = 'on',\n",
    "                      map_boundaries_colour = 'navy',\n",
    "                      map_coastline_resolution = 'high'\n",
    "            )\n",
    "title = magics.mtext()\n",
    "legend = magics.mlegend(legend_display_type = \"continuous\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load predefined ECMWF contour setting:\n",
    "Setting the parameter **contour_automatic_setting** to **'ecmwf'** will give us style developed at ECMWF for this parameter.  \n",
    "\n",
    "More about contour settings in Magics can be found [here](https://confluence.ecmwf.int/display/MAGP/Contouring \"Contouring\")  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 152216 characters]\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "contour = magics.mcont(contour_automatic_setting = 'ecmwf')\n",
    "\n",
    "magics.plot(projection, data, contour, legend, coast, title)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How does Magics know which default style to apply?\n",
    "\n",
    "Magics scans netcdf metadata and looks for standard_name and long_name. If more styles are found, Magics will choose the one with more keys matched. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "We are plotting  r  and it has standard_name =  relative_humidity\n",
      "We are plotting  r  and it has long_name =  Relative humidity\n"
     ]
    }
   ],
   "source": [
    "if 'standard_name' in ds[variable_name].attrs:\n",
    "    print('We are plotting ', variable_name, ' and it has standard_name = ', ds[variable_name].attrs['standard_name'])\n",
    "if 'long_name' in ds[variable_name].attrs:\n",
    "    print('We are plotting ', variable_name, ' and it has long_name = ', ds[variable_name].attrs['long_name'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some ERA5 parameters (that have representation in ecCharts) have more than one built in predefined styles.  \n",
    "Run the next cell and choose a style."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8c1d559cfe4446bfb7ba2064828de15b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Dropdown(description='Choose a style:', options=('sh_grnblu_f65t100i15', 'sh_grnblu_f65t100i15_light', 'ct_grn…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "styles = magics.wmsstyles(data, )\n",
    "all_styles = []\n",
    "for i in styles['styles']:\n",
    "    all_styles.append(i[\"name\"])\n",
    "\n",
    "One_style = widgets.Dropdown(options=all_styles, value = all_styles[0], description='Choose a style:', style=style)\n",
    "display(One_style)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To use this new feature we need to set the parameter **contour_automatic_setting** to **'style_name'**, and add a new parameter **contour_style_name** with the chosen style. In this example it takes value choosen by the previous menu."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 140908 characters]\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "contour = magics.mcont(contour_automatic_setting = 'style_name',\n",
    "                      contour_style_name = One_style.value,\n",
    "                      legend = 'on')\n",
    "\n",
    "magics.plot(projection, data, contour, coast, legend, title)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Magics offers number of predefined palettes, for a bit more flexibility\n",
    "\n",
    "It would be ideal if our number of levels matched number of colours in palette.  \n",
    "If for any reason that is not the case, we can set parameter **contour_shade_palette_policy** to **'lastone'** or **'cycle'**, so Magics knows what to do if there are more levels than colours.   \n",
    "Full list of Magics palletes can be found [here](https://confluence.ecmwf.int/display/MAGP/Predefined+palettes+in+Magics \"Magics palettes\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 198256 characters]\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "contour = magics.mcont(\n",
    "    legend = 'on',\n",
    "    contour = 'off',\n",
    "    contour_shade = 'on',\n",
    "    contour_label = 'off',\n",
    "    contour_level_selection_type = 'level_list',\n",
    "    contour_level_list = [20.00, 30.00, 40.00, 50.00, 60.00, 70.00, 80.00, 90.00, 150.00],\n",
    "    contour_shade_method = 'area_fill',\n",
    "    contour_shade_palette_name = 'm_blue_green_8',\n",
    "    contour_shade_colour_method = 'palette'\n",
    ")\n",
    "\n",
    "magics.plot(projection, data, contour, legend, coast, title)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### If we want, we can have a full control, of course"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 201608 characters]\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "contour = magics.mcont(legend = \"on\",\n",
    "                       contour = \"off\",\n",
    "                       contour_label = \"off\",\n",
    "                       contour_hilo = \"off\",\n",
    "                       contour_level_selection_type = 'level_list',\n",
    "                       contour_level_list = [20.00, 30.00, 40.00, 50.00, 60.00, 70.00, 80.00, 90.00, 150.00],    \n",
    "                       contour_shade = \"on\",\n",
    "                       contour_shade_method = \"area_fill\",\n",
    "                       contour_shade_colour_method = \"list\",\n",
    "                       contour_shade_colour_list = ['#f7fcfd','#e5f5f9','#ccece6',\n",
    "                                                    '#99d8c9','#66c2a4','#41ae76',\n",
    "                                                    '#238b45','#005824'])\n",
    "\n",
    "magics.plot(projection, data, contour, legend, coast, title)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
