V
Python notes (Von P. Walden)
>
Getting Started
>
Installing a python distribution
>
Install Anaconda Python - it’s the best !!
>
Mac/Linux
>
Make sure that you download the latest version (3.X) from the Anaconda python website.
>
Windows
>
Make sure that you download the version 2.7 from the Anaconda python website.
*
Currently the valuable mapping package, basemap, is NOT available in Python 3.X for Windows.
>
Note about Windows computers
>
As of 30 September 2015, “conda install basemap” does not work on computers running Windows under Anaconda Python 3.4.
*
Tried installing via executable file at Sourceforge, but that didn’t work because it is for Python 3.3.
>
Extra Python modules that are useful
>
Essential packages to install manually
*
conda install basemap
*
conda install netCDF4
*
conda install jupyterlab
*
conda install ephem
>
conda install folium
*
folium is a module that creates HTML maps.
>
conda install mplleaflet
>
mplleaflet
*
anaconda search -t conda mplleaflet
*
anaconda show conda-forge/mplleaflet
*
conda install --channel https://conda.anaconda.org/conda-forge mplleaflet
>
conda install xarray
>
xray is a module for handling labeled, N-D arrays.
>
conda install oct2py
>
oct2py
*
anaconda search -t conda oct2py
*
anaconda show IOOS/oct2py
*
conda install --channel https://conda.anaconda.org/IOOS oct2py
*
conda install pydap
*
conda install pandas-datareader
*
conda install pyresample
>
conda install ipyleaflet
>
ipyleaflet
*
anaconda search -t conda ipyleaflet
*
anaconda show conda-forge/ipyleaflet
*
conda install --channel https://conda.anaconda.org/conda-forge ipyleaflet
*
conda install altair
>
conda install ipyvolume
>
conda install metpy
>
conda install climlab (currently only available for Python 2…)
>
geopandas
*
conda install -c conda-forge fiona shapely pyproj rtree
>
conda install -c conda-forge geopandas
>
pip install pysplit
*
conda install geopy
>
Non-equispaced fast Fourier transform (NFFT)
>
pip install nfft
*
Non-uniform fast Fourier transform (pynufft)
>
pip install pynufft
>
conda install -c condo-forge wrf-python
>
conda install jupyter_dashboards -c conda-forge
>
Then execute this code from command line:
*
jupyter nbextension enable jupyter_dashboards--py --sys-prefix
>
Documentation
>
conda install qgrid
>
pip install tweepy
>
Useful packages to install manually
>
PyCWT - wavelet module
*
python setup.py install (after downloading file from https://github.com/regeirk/pycwt)
*
conda install spyder=3.0.2
>
conda install gdal
*
this module may be messing with matplotlib…
*
conda install -c conda-forge glueviz
>
plotly offline graphics
*
conda install --channel https://conda.anaconda.org/conda-forge plotly
>
cufflinks to link pandas with plotly
*
conda install --channel https://conda.anaconda.org/bioconda cufflinks
>
conda install --channel https://conda.anaconda.org/conda-forge ggplot
*
Plotting package by yhat (makers of Rodeo IDE)
>
cartopy
*
conda install --channel https://conda.anaconda.org/conda-forge cartopy
*
conda install holoviews
>
geoviews
*
conda install --channel https://conda.anaconda.org/ioam geoviews
>
fastkml
*
pip install fastkml
>
pendulum - Python datetimes made easy
*
conda install --channel https://conda.anaconda.org/ericmjl pendulum
>
pip install calmap
*
creates nice heat maps from pandas time series
*
conda install seaborn
>
geojson
*
anaconda search -t conda geojson
*
anaconda show conda-forge/geojson
*
conda install --channel https://conda.anaconda.org/conda-forge geojson
>
pyowm - Python API to Open Weather Map (http://openweathermap.org/api)
>
pip install pyowm
*
Currently not available in conda for OS X.
>
Must first obtain on API key from OWM.
*
c4aed96f728ad4d96a4032e2f7db68f3
>
pymiecoated
*
Download and decompress folder.
>
Navigate to folder, then type:
*
python setup.py install
>
Others, but not installed
*
conda install pysolar
*
conda install --channel https://conda.anaconda.org/osgeo fastkml
*
for reading kml files
>
Difficult module installations
>
conda install pygrib
*
But unfortunately is not currently available for mac os x and linux in Python 3.4
>
BUT this can be installed in the ‘py27’ environment using
*
source activate py27
*
conda install --channel https://conda.anaconda.org/jjhelmus pygrib
>
BUT this environment variable must be set before pygrib will work in the “py27” environment.
*
export GRIB_DEFINITION_PATH=/Users/vonw/anaconda/envs/py27/share/grib_api/definitions
>
pygrib
*
conda install proj.4
>
jasper
*
download jasper source code from http://www.ece.uvic.ca/~frodo/jasper/#download
>
install jasper by following instructions in User’s Manual http://www.ece.uvic.ca/~frodo/jasper/ jasper.pdf
*
cd /Users/vonw/Downloads/jasper-1.900.1
*
./configure
*
make
*
make install
*
Seems to have worked, but executables were installed in /Users/vonw/Downloads/jasper-1.900.1/src/appl
>
GRIB API
>
didn’t work… ???
*
grib_api-1.14.0-darwin14.4.0-x86_64-build1.tar.gz
>
Tried this instead
*
Downloaded grib_api-1.14.0-darwin14.4.0-Source.tar
*
tar -xzf  grib_api-1.14.0-Source.tar.gz
*
> mkdir build ; cd build
*
> cmake  ../grib_api-1.14.0-Source  -DCMAKE_INSTALL_PREFIX=/path/to/where/you/install/gribapi
>
pip install pygrib
>
BUT I received this error:
*
nuia:~ vonw$ python
*
Python 3.4.3 |Anaconda 2.3.0 (x86_64)| (default, Mar 6 2015, 12:07:41)
*
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
*
Type "help", "copyright", "credits" or "license" for more information.
*
>>> import pygrib
*
Traceback (most recent call last):
*
File "<stdin>", line 1, in <module>
*
ImportError: dlopen(/Users/vonw/anaconda/lib/python3.4/site-packages/pygrib.so, 2): Library not loaded: libpng16.16.dylib
*
Referenced from: /Users/vonw/anaconda/lib/python3.4/site-packages/pygrib.so
*
Reason: image not found
*
>>> exit()
*
sudo ln -s /Users/vonw/anaconda/lib/libpng16.16.dylib /usr/local/lib/libpng16.16.dylib
*
pip uninstall pygrib
>
pip install pygrib
*
Installed it fresh again, just to make sure.
>
Different Anaconda environments
>
To change environments from the default 3.4 to 2.7, open a shell and type:
*
source activate py2
>
To deactivate and return to Python 3.4, type:
*
source deactivate
>
Differences between Python 2 and Python 3
>
From Aaron Meurer
*
python3-presentation.pdfpython3-presentation
>
Learning Python
>
Style Guide for Python Code (PEP8)
>
A Whirlwind Tour of Python, by Jake VanderPlas
*
a-whirlwind-tour-of-python-2.pdfa-whirlwind-tour-of-python-2
>
Python Data Science Handbook, by Jake VanderPlas
>
Python Quick Reference by Data School
>
Scientific Python Lecture Notes (scipy)
>
SciPy Lectures
*
ScipyLectures-simple.pdfScipyLectures-simple
>
Learning Python
>
0) Learning Python
>
1) Learning Interactive Python (IPython)
>
You’ll want to use the IPython shell instead of a regular Python shell (which is a pain). Read through the first couple chapters of “Learning IPython for Interactive Computing and Data Visualization”, which is attached. It has great information on the basics of IPython.  IPython will be installed along with the Anaconda python distribution.  Note that this book also has some good chapters of computing using IPython, but it assumes you already know Python.  Therefore, I suggest you move onto step 3 first.
>
2) Data Analysis using Python
>
Then start working your way through JR Johanssen’s lectures on Python; PDF is also attached (http://nbviewer.ipython.org/github/jrjohansson/scientific-python-lectures/tree/master/). Chapter 1 gives a nice and concise introduction to Python programming. Subsequent chapters explain how to use Python for data analysis, including Chapter 5 on matplotlib which is the standard graphics package.  (For the future, see Chapter 6 on how to easily interface Python with Fortran (and C)).
*
Scientific-Computing-with-Python.pdfScientific-Computing-with-Python
>
An alternative is to look through the lectures on SciPy
>
3) My own tips
>
A) Everything in Python is an “object”, so learn how to query them.
*
help(object)
*
dir(object)
>
B) Learn about the common Python objects.
*
Integers
*
Floats
*
Booleans
*
Lists (and Tuples)
>
Dictionaries
*
Make sure that you understand what Lists and Dictionaries are in Python. (Tuples are very similar to Lists.) These are described in Section 2.6 in Johanssen’s tutorial. These are essential building blocks in Python. For instance, arrays in Numerical Python (numpy) are like Lists, and netCDF files usually behave like Dictionaries. This isn’t very important for you now, but it might be later!
*
And numpy arrays later !!
>
C) Parentheses matter in Python; use them to understand what an object is.
>
Round vs. square vs. “squiggle”
*
This is quick way to understand what you can do with an object.
>
D) Also get used to using type(object).
*
This is an alternative to understanding parentheses.
*
E) Substitute comprehensions for for loops whenever possible (because they’re compact and slick).
>
A gallery of interesting Jupyter and IPython notebooks
>
Jupyter Notebook
>
Once a basic competency level is reached in Python, it might be useful to start using Jupyter notebooks.
>
nbviewer - a simple way to share Jupyter notebooks
>
Language Kernels
*
Octave
>
pandas
>
Reproducible Data Analysis
>
Learn Pandas (especially for time series analysis)
*
McKinney 2015.pdfMcKinney 2015
>
pandas cookbook
*
McKinney, W. (2012). Python for data analysis (pp. 1-470).
>
Things in Pandas I wish I’d Known Earlier
>
Intermediate
>
0) Intermediate Python (21 tips)
>
1) Look at the useful examples from EarthPy
>
2) Learn Pandas (especially for time series analysis)
*
McKinney, W. (2012). Python for data analysis (pp. 1-470).
>
Advanced
*
Lanaro, G. (2013). Python high performance programming. Packt Publishing. Retrieved from www.packtpub.com.
>
xarray tutorial (UW eScience Institute)
>
Google Earth Engine
>
Videos
>
Anaconda YouTube Channel
>
YouTube lectures by Lorena Barba
>
Speeding up Python
>
On iMac
>
How to activate all cores from Python
>
numba
>
Numba: High-Performace Python with CUDA Acceleration
>
Numba Tutorials (in Jupyter Notebooks)
>
Example scripts that use numba
>
ipyparallel
>
cython
>
CUDA
>
CUDA installation on a Mac
>
CUDA-supported GPUs
>
Steps to install on a computer
>
Download and install Anaconda Python.
*
conda update conda
*
conda install numba
*
conda install cudatoolkit
>
GPU Accelerated Computing with Python
>
Seven Things You Might Not Know about Numba
>
CUDA Casts by Nvidia
>
dask
>
Graphics
>
Mike Bostock !!
>
colormaps
>
The new default colormap for matplotlib is “viridis”.
>
matplotib
>
Creating Interactive Graphics in Matplotlib (Jake VanderPlas)
>
bokeh
*
Great package for creating web graphics; very useful inside of Jupyter notebooks.
>
Jupyter notebook on Bokeh
>
Very useful hints
>
Working in IPython notebooks (including “interactors”)
>
NCAR graphics
>
rasterio
*
Rasterio: access to geospatial raster data — rasterio 1.0a9 documentation
>
GitHub - conda-forge/rasterio-feedstock: A conda-smithy repository for rasterio.
*
conda config --add channels conda-forge
*
conda install rasterio
>
lightning
>
Jupyter notebook
>
folium
*
folium is a python wrapper for leaflet.js, a javascript mapping platform.
>
Useful web links
>
vispy
>
Books
>
seaborn
*
High-level graphics package for drawing statistical graphics.
>
plotly
>
satellite imagery
>
pygaarst
>
GeoTIFF
>
gdal
>
MODIS
>
Simple script to read data (after conversion using h4toh5)
*
import gdal
*
ds = gdal.Open('/Users/vonw/data/modis/MOD13C2.A2015032.005.2015070071457.h5')
*
subds = [sd for sd, descr in ds.GetSubDatasets()]
*
print(subds)
*
# Then choose a sub-dataset from the listing of subds
*
# for instance…
*
ds2 = gdal.Open(subds[6]) # NDVI map
*
data = ds2.ReadAsArray()
*
imshow(data)
>
mpld3
>
Pythonic Perambulations
*
 
>
mapping
>
cartopy
>
vega and vincent
>
References
>
IPython notebooks
>
Earth Science and Geospatial Data
>
Books
>
O’Reilly Publishing (oreilly.com)
*
Lutz, M. (2011). Learning python (pp. 1-1216).
*
Martelli, A. (2010). Python in A nutshell (pp. 1-736).
*
Lutz, M. (2011). Programming python (pp. 1-1628).
*
Downey, A. (2012). Think python. Green Tea Press. Retrieved from http://www.greenteapress.com/thinkpython/
>
Packt Publishing (packtpub.com)
*
IPython Interactive Computing and Visualization Cookbook
>
Robert Johansson
*
Numerical Python - A practical techniques approach for industry
>
Conferences
>
AnacondaCon
>
JupyterCon
>
PyData
>
SciPy
>
EuroSciPy
>
Cheat Sheets
>
Python Cheat Sheets
>
Scientific Python
>
Pandas
>
Bokeh
>
Other
>
Reproducible Research using Jupyter
>
From Jake VanderPlas, U. Washington, eScience Institute
>
Version Control
>
GIT - “Global Information Tracker”
*
McCullough, J. L. A. M. (2012). Version control with git (pp. 1-454). Retrieved from http://www.google.com/search?client=safari&rls=10_7_4&q=Version+Control+with+Git&ie=UTF-8&oe=UTF-8
>
Education
>
Difficulty deploying JupyterHub for a large class
>
Cookbooks
>
HDF
>
h4toh5
>
This is an important utility for converting files in HDF4 format (such as HDF-EOS) to HDF5 format, so that they can be easily read into Python using the h5py or gdal modules.
>
Simple to use:
>
h4toh5 /Users/vonw/data/modis/MOD13C2.A2015032.005.2015070071457.hdf
*
This creates /Users/vonw/data/modis/MOD13C2.A2015032.005.2015070071457.h5, which can be read by gdal or h5py.
>
Installation
>
I downloaded this from http://hdfeos.org/software/tool.php#H4H5TOOLS and just copied it to /usr/local/bin/h4toh5.
*
It runs from the command line as h4toh5 because /usr/local/bin is on the path.
>
Computational Tools for Mac
>
Computational Fluid Dynamics (CFD)
>
The OpenFOAM® Foundation
>
CFD Python- 12 steps to Navier-Stokes -- Lorena A. Barba Group
>
SfePy- Simple Finite Elements in Python
>
Geographic Information Systems (GIS)
>
Important software packages for GIS (but not necessarily Python)
>
OSGeo - Open Source Geospatial Foundation
>
osgeo4w (for Windows)
*
Alternative to ArcGIS and QGIS
>
GEOS - Geometry Engine, Open Source
>
Mapnik
>
GDAL - Geospatial Data Abstraction Library
*
Historically GDAL (raster) and OGR (vector) were separate projects, but with GDAL v2.0, these projects are now tightly coupled into GDAL.
>
Geospatial Packages/Modules in Python
>
GDAL 2.0.1 : Python Package Index
>
Downsampling with GDAL in python
>
Geocoding
*
geocoder
*
To install on Mac
*
pip install pyproj
*
pip install geocoder
>
Tutorials
>
Google Earth Engine
>
Installation
>
Steps on Mac (from a terminal window)
>
0) source activate py27
>
Looks like earthengine requires Python 2.7.
>
I’m getting an error in Python 3.5 when I try to import ee.mapclient
*
ImportError: No module named 'cStringIO'
*
This is related to an upgrade between Python 2 and Python 3…
*
1) pip install google-api-python-client
*
2) python -c "from oauth2client import crypt"
>
3) openssl version
*
Just to make sure that openssl is installed; it is!
*
4) pip install earthengine-api
>
5) earthengine authenticate
>
Pops up a web page that requests that you allow Google to handle your requests to Earth Engine. Press “Allow” pops up a window
*
Copy the authentication string back into your terminal window, because “earthengine authenticate” expects this.
>
6) Then you should be good to go; “import ee” from python.
>
See example Python scripts at
>
Tutorials to learn how to use Google’s Code Editor (using JavaScript)
>
Sample scripts that are cool!
>
Image: Hillshade
*
This is Mount Rainier
>
Image: Landcover Cleanup
*
Adjust date
*
Also comment out the smoothing.
>
Image Collection: Modis Cloud Masking
*
Show how to switch off layers.
*
Adjust dates to show snow
*
Arrays: Quality Mosaic
>
User Interface: Collection Slider
*
Check out northern Norway
>
Python versus R
>
Numpy for Matlab Users
>
NumFOCUS
>
Social Media
>
Twitter
*
conda install --channel https://conda.anaconda.org/asmeurer twitter
*
conda install --channel https://conda.anaconda.org/asmeurer twitterapi
>
Eat Python for lunch?
>
Future Ideas
>
Statistical Analysis
>
Examples
>
Data
>
Videos
>
Time Series Analysis
>
From pandas
*
dir(pd.stats)
*
dir(pd.stats.api)
*
help(pd.stats.ols)
>
Examples to use
>
Good example using Arctic Oscillation dataset
>
S&P 500
>
Variograms
*
Talk to Tsengel
*
Using Jupyter Notebooks for reproducible research and collaboration (documentation)
*
Mining Social Media using Python
*
Web Scaping
*
Symbolic Python
*
Machine Learning
>
Fall 2016
>
Jupyter Notebooks
>
15 September 2016
*
PythonLunch_20160915.htmlPythonLunch_20160915
*
PythonLunch_20160915.ipynbPythonLunch_20160915
>
Ideas
*
GIS
>
Computational Speed
*
ipyparallel
*
numba
*
web scraping
>
data mining
*
twitter analysis
>
HTML graphics
*
bokeh
*
rBokeh !!
>
Summer 2016
>
24 May
>
Monty Python
>
Confuse-A-Cat
*
Rick Rupp’s presentation
>
Von
>
GIS programs typically have their OWN PYTHON ENVIRONMENT
*
They don’t use your “normal” environment.http://www.qgistutorials.com/en/
>
ESRI
>
ArcPy
>
QGIS
>
Tutorials
>
Future Ideas
*
Mining Social Media using Python
*
Statistical Analysis
*
Machine Learning
*
Time Series Analysis
*
Variograms
*
Talk to Tsengel
*
Using Jupyter Notebooks for reproducible research and collaboration (documentation)
>
17 May
>
Monty Python
*
 
>
Review
*
Everything in Python is an object
*
Main types
*
strings
*
integers
*
floating-point numbers
*
lists
*
tuples
*
dictionaries
*
numpy adds:
*
arrays
*
Pandas adds:
*
Series
*
DataFrames
*
Panels
*
Flow Control
*
for loops
*
if-else
*
while
*
break, continue, pass
>
What is GIS?
>
GIS = Geographic Information System
>
“Word processor” for geographic information
>
Two main types of features (that are located geographically)
*
raster
*
regular grids (arrays) of data
>
vector
*
points, lines, polygons
>
attribute tables
*
attach information to features
>
Example GIS programs
>
ArcGIS
>
GRASS
>
QGIS
*
Python and GIS
>
Spring 2016
>
10 May
*
Instruments
*
Internet of Things (IoT)
*
Computers
*
Interfaces
*
RS-232 - Standard Serial
*
TXD / RXD
*
I2C - Inter-Integrate Circuit
*
SPI - System Packet Interface
*
LabView
*
Sensors
*
Example - Bosch BMP180
*
IoT for Air Quality
*
CO2 sensor
*
Alphasense
*
VOCs
*
Hamamatsu micro-spectrometer
*
Demo
>
26 April
>
Monty Python
>
Argument Clinic
>
Review
>
Simple graphics using matplotlib
>
simple plot
*
figure()
*
plot(randn(1000,1),'o')
*
xlabel('Number')
*
ylabel('Value')
*
title('simple plot demo')
>
demonstrate subplot
*
figure()
*
subplot(2,1,1)
*
plot(randn(1000,1),'o')
*
ylabel('Value')
*
title('simple subplot demo')
*
subplot(2,1,2)
*
plot(rand(1000,1),'o')
*
ylabel('Value')
*
xlabel('Number')
>
Just a bit more on 3D graphics
>
mplot3d in matplotlib
>
Reminder about folium
>
Mt. St. Helens
>
From within a Jupyter notebook
*
import folium
*
mtsthelens = folium.Map(location=[46.1991, -122.1889])
*
mtsthelens
>
basemap - matplotlib extension
>
Setting up the map
>
Plotting data on a map
>
Example of Jupyter notebook to plot GFS data over NH
*
from pylab import *
*
from netCDF4 import Dataset
*
from mpl_toolkits.basemap import Basemap, addcyclic
*
import matplotlib.pyplot as plt
*
import numpy as np
*
from datetime import datetime, timedelta
*
import sys
*
 
*
# Determine nearest forecast time.
*
dn = datetime.utcnow() - timedelta(0.25)
*
yymmdd = dn.strftime('%Y%m%d')
*
ctime = int(dn.strftime('%H%M'))
*
ftimes = array([0, 600, 1200, 1800])
*
hhmm = str(ftimes[ max( find(ftimes < ctime) ) ]).zfill(4)
*
print('Date/Time: ', yymmdd, ' ', dn.strftime('%H%M').zfill(4), ' UTC')
*
 
*
print('Opening connection to thredds.ucar.edu...')
>
try:
*
gfs = Dataset(fn)
>
except:
*
print('Data is NOT currently available from http://thredds.ucar.edu/thredds/dodsC/grib/NCEP/GFS/Global_0p5deg/GFS_Global_0p5deg_' + yymmdd + '_' + hhmm + '.grib2/GC')
*
sys.exit()
*
 
*
# The "gfs" reference in python is a set of nested dictionaries.
*
 
*
# Typing "gfs" at the python prompt will list the available variables.
*
# However, it is more convenient to list the variables from the web:
*
 
*
# List the contents of a single variable like:
*
# gfs.Temperature_surface
*
# This object is a dictionary with four variables: lat, lon, time, Temperature_surface
*
 
*
# Date and time
*
print(' Loading date and time...')
*
time_offset = fn[-13:-9]
*
date = fn[-22:-14]
*
time = gfs.variables['time'][:]
*
 
*
# Lat/lon
*
print(' Loading latitude and longitude...')
*
lat = gfs.variables['lat'][:]
*
lon1 = gfs.variables['lon'][:]
*
# Surface Temperature
*
print(' Loading surface temperature...')
*
Ts = gfs.variables['Temperature_surface'][0,:] - 273.15
*
Ts, lon = addcyclic(Ts, lon1)
*
# Pressure reduced to mean sea level
*
print(' Loading surface pressure...')
*
Pmsl = gfs.variables['Pressure_reduced_to_MSL_msl'][0,:]/100. # in mb
*
Pmsl, lon = addcyclic(Pmsl, lon1)
*
# Masked pressure data to prevent contour labels being printed outside of round map area.
*
#Pmsl = np.ma.masked_array(Pmsl, mask=lat>-40)
*
#Ptrop = gfs.variables['Pressure_tropopause'][0,:]/100. # in mb
*
#gph = gfs.variables['Geopotential_height'][0,:]
*
# Thickness of Geopotential Height between 1000 and 500 mb
*
#print(' Loading 500-mb geopotential height...')
*
#gph_1000mb = gfs.variables['Geopotential_height'][0,0,:,:]
*
#gph_500mb = gfs.variables['Geopotential_height_isobaric'][0,12,:,:]
*
#thickness_1000_500 = gph_500mb - gph_1000mb
*
# Surface winds
*
#print(' Loading surface wind components...')
*
#u = gfs.variables['U-component_of_wind'][0,0,:,:]*1.94 # Convert from m s-1 to knots
*
#v = gfs.variables['V-component_of_wind'][0,0,:,:]*1.94
*
 
*
# Plot map
*
print(' Creating map...')
*
figure(figsize=(14,14))
*
 
*
m = Basemap(projection='npstere', lon_0=-105., lat_0=45., boundinglat=45., resolution='l')
*
#m = Basemap(projection='npstere', lon_0=-105., lat_0=45., boundinglat=45., resolution='l', round=True)
*
x,y = m(*np.meshgrid(lon,lat))
*
 
*
# Pmsl or Geopotential thickness
*
CS=m.contour(x,y,Pmsl,range(920,1040,5),colors='k')
*
plt.clabel(CS, CS.levels,inline=1,fmt='%4.0f',fontsize=12) # label every second level
*
#CS=m.contour(x,y,gph_500mb,range(4000,6000,200),colors='k')
*
#plt.clabel(CS, CS.levels,inline=1,fmt='%4.0f',fontsize=12) # label every second level
*
 
*
# Surface temperature
*
m.contourf(x,y,Ts,range(-60,61,1),extend='both',cmap='RdBu_r')
*
CB=plt.colorbar(ticks=range(-60,61,20), shrink=0.6)
*
 
*
# Surface wind barbs
*
#m.barbs(x[0::5,0::10],y[0::5,0::10],u[0::5,0::10],v[0::5,0::10],length=5)
*
 
*
# draw coastline, parallels and meridians.
*
m.drawcoastlines()
*
m.drawparallels(range(50,90,10))
*
m.drawmeridians(range(-180,180,10))
*
 
*
# Plot the location of Summit Station.
*
sx, sy = m(-38.,72.)
*
m.scatter(sx,sy,s=40,c=[1,1,1],marker='o')
*
 
*
plt.text(3500000,-500000,'Surface T (C) and P (mb)')
*
plt.title('GFS model forecast for ' + date + ' at ' + time_offset + ' UTC')
*
 
*
plt.show()
>
12 April
>
Monty Python
>
Review
*
Missing values in NumPy and Pandas
>
Graphics
*
Matplotlib
>
Very similar to Matlab
*
r = randn(1000,1)
*
plot(r)
*
xlabel(‘Count Number’)
*
ylabel(‘Random Value’)
*
title(‘1000 Random Numbers’)
>
Mayavi
*
Must install under a Python 2.7 distribution; not currently available for Python 3.x.
>
Quick demo
>
bokeh
*
import matplotlib.pyplot as plt
*
import numpy as np
*
 
*
from bokeh import mpl
*
from bokeh.plotting import output_file, show
*
 
*
r = randn(1000,1)
*
plt.plot(r)
*
plt.xlabel(‘Count Number’)
*
plt.ylabel(‘Random Value’)
*
plt.title(‘1000 Random Numbers’)
*
 
*
output_file(‘/Users/vonw/Desktop/bokeh_plot.html’)
*
 
*
show(mpl.to_bokeh())
>
folium
>
Seaborn
>
5 April 2016 - More on NumPy and Scipy
>
Silly Olympics
*
Review
>
Pandas
*
time index is easy
*
iaq.index.weekday
*
iaq.index.week
*
iaq.index.month
*
iaq.index.day
*
groupby function
*
iaq.groupby(‘Type’).describe()
*
Indexing a particular day
*
fruit[‘2014-08-31’:‘2014-08-31’]
>
Namespace (and Scope)
>
From “Learning Python”, 4th edition, by Mark Lutz
*
pastedImage
>
The LEGB Rule
*
pastedImage0
>
Examples
*
X = 99
>
def func():
*
X = 88
*
print(X)
*
func()
*
print(X)
*
pastedImage2
>
What is the deal with %pylab in IPython?
*
pastedImage1
>
SciPy cookbook
>
Examples
*
interpolation.py
*
linearRegression.py
*
SOI.py
*
PearsonCorrelation.py
>
29 March 2016 - Numpy and Scipy
>
Monty Python - Silly Walks
>
Scipy Lecture Notes
>
Scipy
>
Numpy
>
22 March 2016 - More on Data Analysis
>
Monty Python’s Bicycle Repairman
>
Reading data with pandas
*
working with text files
*
working with csv files
*
working with Excel spreadsheets
*
working with netCDF files
>
8 March 2016 - Using Python for data analysis
>
Monty Python’s philosopher’s football
>
JR Johannsen’s Scientific Python Lectures
>
Contains JR Johannsen’s Jupyter notebooks !
>
pandas
>
1 March 2016 - The Basics
>
Dead Parrot
>
0) Why Python?
*
Common language for many tasks
*
Free and open source
*
Good replacement for Matlab
*
General purpose
*
Works well with other environments.
*
linux, octave, (and R)
*
What do I use it for?
*
Data analysis
*
Data visualization
*
Geospatial analysis (ArcGIS, QGIS)
*
Remote sensing
*
Instrumentation
*
Simple interface with Raspberry Pi
*
Incredible resources for research and teaching
*
IPython
*
Jupyter notebooks
*
What are your needs?
*
More videos !!
*
whole movies
*
Syntax
*
Modules / Packages
*
Transitioning from Matlab
*
Geospatial
*
Rick Rupp
*
Version control (git)
>
1) Getting Started
*
Packages / Modules
*
conda
>
2) Basics
*
Python data types (objects)
*
integers
*
floating point numbers
*
booleans
*
list (tuples)
*
dictionaries
*
Make sure that you understand what Lists and Dictionaries are in Python. (Tuples are very similar to Lists.) These are described in Section 2.6 in Johanssen’s tutorial. These are essential building blocks in Python. For instance, arrays in Numerical Python (numpy) are like Lists, and netCDF files usually behave like Dictionaries. This isn’t very important for you now, but it might be later!
*
And numpy arrays later !!
*
pandas Series and DataFrames (similar to R)
>
3) My own tips
*
A) Everything in Python is an “object”, so learn how to query them.
*
help(object)
*
dir(object)
*
B) Learn (and use) about the common Python objects.
*
C) Parentheses matter in Python; use them to understand what an object is.
*
Round vs. square vs. “squiggle”
*
This is quick way to understand what you can do with an object.
*
D) Also get used to using type(object).
*
This is an alternative to understanding parentheses.
*
E) Substitute comprehensions for for loops whenever possible (because they’re compact and slick).
>
My Python notes
>
Eat Python for lunch
>
Monty Python
*
Telephone repair man
*
New Cooker Sketch
*
Gumby Brain Surgeon
*
Future seminars
*
26 April 2016 - Working with Instruments