Project: Investigate a Movie Dataset

Table of Contents

Introduction

I've selected TMDb movie data for my analysis.

Apart from entertainment to audience - making a movie involves lot of decision to be taken in order to make it successful, when we say successful we mean making profit - popular movies in terms of vote_average can be related to being successful as well.

Below are the questions we will be answering by exploring the dataset:

  1. Which production company makes successful movies
  2. Which genre movies are more successful
  3. What is the right runtime to have
  4. Which month to release the movie
  5. How much to invest based on the genre
In [268]:
import numpy as np
import pandas as pd
from datetime import datetime
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline

Data Wrangling

General Properties

In [269]:
# using pandas to load csv file
df = pd.read_csv('tmdb-movies.csv')
df.head()
Out[269]:
id imdb_id popularity budget revenue original_title cast homepage director tagline ... overview runtime genres production_companies release_date vote_count vote_average release_year budget_adj revenue_adj
0 135397 tt0369610 32.985763 150000000 1513528810 Jurassic World Chris Pratt|Bryce Dallas Howard|Irrfan Khan|Vi... http://www.jurassicworld.com/ Colin Trevorrow The park is open. ... Twenty-two years after the events of Jurassic ... 124 Action|Adventure|Science Fiction|Thriller Universal Studios|Amblin Entertainment|Legenda... 6/9/15 5562 6.5 2015 1.379999e+08 1.392446e+09
1 76341 tt1392190 28.419936 150000000 378436354 Mad Max: Fury Road Tom Hardy|Charlize Theron|Hugh Keays-Byrne|Nic... http://www.madmaxmovie.com/ George Miller What a Lovely Day. ... An apocalyptic story set in the furthest reach... 120 Action|Adventure|Science Fiction|Thriller Village Roadshow Pictures|Kennedy Miller Produ... 5/13/15 6185 7.1 2015 1.379999e+08 3.481613e+08
2 262500 tt2908446 13.112507 110000000 295238201 Insurgent Shailene Woodley|Theo James|Kate Winslet|Ansel... http://www.thedivergentseries.movie/#insurgent Robert Schwentke One Choice Can Destroy You ... Beatrice Prior must confront her inner demons ... 119 Adventure|Science Fiction|Thriller Summit Entertainment|Mandeville Films|Red Wago... 3/18/15 2480 6.3 2015 1.012000e+08 2.716190e+08
3 140607 tt2488496 11.173104 200000000 2068178225 Star Wars: The Force Awakens Harrison Ford|Mark Hamill|Carrie Fisher|Adam D... http://www.starwars.com/films/star-wars-episod... J.J. Abrams Every generation has a story. ... Thirty years after defeating the Galactic Empi... 136 Action|Adventure|Science Fiction|Fantasy Lucasfilm|Truenorth Productions|Bad Robot 12/15/15 5292 7.5 2015 1.839999e+08 1.902723e+09
4 168259 tt2820852 9.335014 190000000 1506249360 Furious 7 Vin Diesel|Paul Walker|Jason Statham|Michelle ... http://www.furious7.com/ James Wan Vengeance Hits Home ... Deckard Shaw seeks revenge against Dominic Tor... 137 Action|Crime|Thriller Universal Pictures|Original Film|Media Rights ... 4/1/15 2947 7.3 2015 1.747999e+08 1.385749e+09

5 rows × 21 columns

In [270]:
df.shape
Out[270]:
(10866, 21)

We have 10,866 dataset meaning data of 10,866 movies and each movie have 21 features.

In [271]:
df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 10866 entries, 0 to 10865
Data columns (total 21 columns):
id                      10866 non-null int64
imdb_id                 10856 non-null object
popularity              10866 non-null float64
budget                  10866 non-null int64
revenue                 10866 non-null int64
original_title          10866 non-null object
cast                    10790 non-null object
homepage                2936 non-null object
director                10822 non-null object
tagline                 8042 non-null object
keywords                9373 non-null object
overview                10862 non-null object
runtime                 10866 non-null int64
genres                  10843 non-null object
production_companies    9836 non-null object
release_date            10866 non-null object
vote_count              10866 non-null int64
vote_average            10866 non-null float64
release_year            10866 non-null int64
budget_adj              10866 non-null float64
revenue_adj             10866 non-null float64
dtypes: float64(4), int64(6), object(11)
memory usage: 1.7+ MB

Looking at the data and info, we can conclude that certain features doesn't really influence the questions that we are trying to answer. We can go ahead and drop those columns.

Data Cleaning

Afte performing some operation on data, we shall check the data by calling df.head(1) and confirm it is working.

In [272]:
del_col = ['id', 'popularity', 'homepage', 'tagline', 'keywords', 'overview', 'vote_count', 'budget_adj', 'revenue_adj']
df.drop(del_col, axis=1, inplace=True)
In [273]:
# check for NaN
df.isnull().sum()
Out[273]:
imdb_id                   10
budget                     0
revenue                    0
original_title             0
cast                      76
director                  44
runtime                    0
genres                    23
production_companies    1030
release_date               0
vote_average               0
release_year               0
dtype: int64

Removing movies which have missing genres

In [274]:
df.dropna(subset=['genres'], inplace=True)

Removing movies which doesn't have imdb_id

later we will splitting movie to different rows based on genres, this allows us to groupby using imdb_id and perform more analysis.

In [275]:
df.dropna(subset=['imdb_id'], inplace=True)
In [276]:
# check for NaN
df.isnull().sum()
Out[276]:
imdb_id                    0
budget                     0
revenue                    0
original_title             0
cast                      75
director                  39
runtime                    0
genres                     0
production_companies    1012
release_date               0
vote_average               0
release_year               0
dtype: int64

Now the data looks good.

But we still need to extract more information from existing feature which might help us analyse data better.

Calculating profit

In [277]:
df['profit'] = df.apply(lambda x: x['revenue']-x['budget'], axis=1)
In [278]:
df.head(1)
Out[278]:
imdb_id budget revenue original_title cast director runtime genres production_companies release_date vote_average release_year profit
0 tt0369610 150000000 1513528810 Jurassic World Chris Pratt|Bryce Dallas Howard|Irrfan Khan|Vi... Colin Trevorrow 124 Action|Adventure|Science Fiction|Thriller Universal Studios|Amblin Entertainment|Legenda... 6/9/15 6.5 2015 1363528810

Extracting month

In [279]:
# converting date to right format
df['release_date'] = pd.to_datetime(df['release_date'])
In [280]:
df['release_month'] = df.apply(lambda x: x.release_date.month, axis=1)
In [281]:
df.head(1)
Out[281]:
imdb_id budget revenue original_title cast director runtime genres production_companies release_date vote_average release_year profit release_month
0 tt0369610 150000000 1513528810 Jurassic World Chris Pratt|Bryce Dallas Howard|Irrfan Khan|Vi... Colin Trevorrow 124 Action|Adventure|Science Fiction|Thriller Universal Studios|Amblin Entertainment|Legenda... 2015-06-09 6.5 2015 1363528810 6

Exploratory Data Analysis

Unique values for each feature in entire dataset

In [282]:
df.nunique()
Out[282]:
imdb_id                 10834
budget                    556
revenue                  4702
original_title          10540
cast                    10690
director                 5054
runtime                   246
genres                   2037
production_companies     7437
release_date             5902
vote_average               71
release_year               56
profit                   5006
release_month              12
dtype: int64

Who directed the most movies

In [283]:
df['director'].value_counts()
Out[283]:
Woody Allen                                                                                                    45
Clint Eastwood                                                                                                 34
Martin Scorsese                                                                                                29
Steven Spielberg                                                                                               29
Ridley Scott                                                                                                   23
Ron Howard                                                                                                     22
Steven Soderbergh                                                                                              22
Joel Schumacher                                                                                                21
Brian De Palma                                                                                                 20
Wes Craven                                                                                                     19
Barry Levinson                                                                                                 19
Tim Burton                                                                                                     19
John Carpenter                                                                                                 18
Mike Nichols                                                                                                   18
Rob Reiner                                                                                                     18
David Cronenberg                                                                                               18
Sidney Lumet                                                                                                   17
Walter Hill                                                                                                    17
Francis Ford Coppola                                                                                           17
Oliver Stone                                                                                                   17
Spike Lee                                                                                                      17
Stephen Frears                                                                                                 17
Robert Zemeckis                                                                                                17
Renny Harlin                                                                                                   17
Tyler Perry                                                                                                    17
Norman Jewison                                                                                                 17
Peter Hyams                                                                                                    17
Stephen Herek                                                                                                  16
Blake Edwards                                                                                                  16
Richard Donner                                                                                                 16
                                                                                                               ..
Pablo Berger                                                                                                    1
Iain B. MacDonald                                                                                               1
Cedric Sundstrom                                                                                                1
Don Argott|Sheena M. Joyce                                                                                      1
Heidi Ewing|Rachel Grady                                                                                        1
Zachary Heinzerling                                                                                             1
Sean Olson                                                                                                      1
Peter Chung|Yoshiaki Kawajiri|Takeshi Koike|Mahiro Maeda|Kôji Morimoto|Shinichiro Watanabe|Andrew R. Jones     1
Franny Armstrong                                                                                                1
Majid Majidi                                                                                                    1
David Zellner                                                                                                   1
Jesse T. Cook                                                                                                   1
Debra Granik                                                                                                    1
Coley Sohn                                                                                                      1
Jean-Baptiste Andrea|Fabrice Canepa                                                                             1
David Lowery                                                                                                    1
Ross Duffer|Matt Duffer                                                                                         1
Andy Hamilton|Guy Jenkin                                                                                        1
Zack Parker                                                                                                     1
Patrick Robert Young|Powell Robinson                                                                            1
Aaron Hann|Mario Miscione                                                                                       1
Andy Palmer                                                                                                     1
Saul Blinkoff|Elliot M. Bour|Robin Steele                                                                       1
Ekachai Uekrongtham                                                                                             1
Enki Bilal                                                                                                      1
Richard Elfman                                                                                                  1
Howard E. Baker|John Fox|Kyungho Lee                                                                            1
Erick Zonca                                                                                                     1
Naomi Foner Gyllenhaal                                                                                          1
Mark Mori                                                                                                       1
Name: director, Length: 5054, dtype: int64

Which production company made most movies

In [284]:
df['production_companies'].value_counts()
Out[284]:
Paramount Pictures                                                                                                                                          156
Universal Pictures                                                                                                                                          133
Warner Bros.                                                                                                                                                 84
Walt Disney Pictures                                                                                                                                         75
Metro-Goldwyn-Mayer (MGM)                                                                                                                                    72
Columbia Pictures                                                                                                                                            72
New Line Cinema                                                                                                                                              61
Touchstone Pictures                                                                                                                                          51
20th Century Fox                                                                                                                                             50
Twentieth Century Fox Film Corporation                                                                                                                       49
TriStar Pictures                                                                                                                                             45
Orion Pictures                                                                                                                                               42
Miramax Films                                                                                                                                                32
Columbia Pictures Corporation                                                                                                                                31
DreamWorks Animation                                                                                                                                         31
Pixar Animation Studios                                                                                                                                      30
Walt Disney Productions                                                                                                                                      29
Dimension Films                                                                                                                                              28
United Artists                                                                                                                                               23
Imagine Entertainment|Universal Pictures                                                                                                                     22
Lions Gate Films                                                                                                                                             21
The Asylum                                                                                                                                                   21
Marvel Studios                                                                                                                                               20
Walt Disney Pictures|Pixar Animation Studios                                                                                                                 17
New World Pictures                                                                                                                                           17
American International Pictures (AIP)                                                                                                                        14
Disney Channel                                                                                                                                               14
Hammer Film Productions                                                                                                                                      13
Walt Disney Pictures|Walt Disney Feature Animation                                                                                                           12
Warner Bros. Pictures                                                                                                                                        12
                                                                                                                                                           ... 
ARP Sélection                                                                                                                                                1
Paramount Pictures|Dino de Laurentiis Cinematografica|BHE Films|Verona Produzione                                                                             1
High Delft Pictures                                                                                                                                           1
Halicki Productions                                                                                                                                           1
Accelerated Matter                                                                                                                                            1
Revelations Entertainment|Dog Pond Productions|Yan Film Group|Paris Film                                                                                      1
Marvel Studios|Marvel Entertainment                                                                                                                           1
Allegro-Film|Filmfonds Wien|ORF Film/Fernseh-Abkommen                                                                                                         1
Buena Vista Distribution Company                                                                                                                              1
Twentieth Century Fox Film Corporation|1492 Pictures|Constantin Film Produktion|Marvel Enterprises|Kumar Mobiliengesellschaft mbH & Co. Projekt Nr. 3 KG      1
Walt Disney Pictures|Walt Disney Animation Australia|Walt Disney Animation Canada|Walt Disney Television Animation                                            1
Laser Unicorns                                                                                                                                                1
Celluloid Dreams|Killer Films|John Wells Productions|Montfort Producciones                                                                                    1
Dark Castle Entertainment|Zinc Entertainment Inc.|Mobicom Entertainment                                                                                       1
Universal Pictures|Dentsu|Relativity Media|Kennedy/Marshall Company, The|Captivate Entertainment                                                              1
Metro-Goldwyn-Mayer (MGM)|Dino De Laurentiis Company|Spelling Films                                                                                           1
Destination Films|Red Wagon Entertainment|Frontera Productions                                                                                                1
World Entertainment                                                                                                                                           1
Lionsgate|Fidélité Films|Hwy61                                                                                                                              1
Dead Old Man Productions|Middle Fork Productions                                                                                                              1
Pathé Pictures International|UK Film Council|Celador Films|Canal+|Warner Bros.                                                                               1
Televisión Española (TVE)|Paraíso|Compañía de Aventuras Comerciales                                                                                      1
New Line Cinema|Illusion Entertainment Group                                                                                                                  1
DNA Films|Ingenious Film Partners|UK Film Council|Moving Picture Company (MPC)                                                                                1
Renaissance Pictures|Pacific Renaissance Pictures Ltd.|Universal Television                                                                                   1
Warner Bros.|Devoted Productions                                                                                                                              1
Toy Gun Films|Touchdown Productions                                                                                                                           1
Bungalow Productions|Silver Lion Films|Vision View Entertainment                                                                                              1
Blinky Productions                                                                                                                                            1
EMI Films Ltd.|Casablanca Filmworks                                                                                                                           1
Name: production_companies, Length: 7437, dtype: int64

Let's see the finance of movies

In [285]:
df['budget'].describe()
Out[285]:
count    1.083500e+04
mean     1.466755e+07
std      3.094749e+07
min      0.000000e+00
25%      0.000000e+00
50%      0.000000e+00
75%      1.500000e+07
max      4.250000e+08
Name: budget, dtype: float64
In [286]:
df['revenue'].describe()
Out[286]:
count    1.083500e+04
mean     3.993726e+07
std      1.171513e+08
min      0.000000e+00
25%      0.000000e+00
50%      0.000000e+00
75%      2.417286e+07
max      2.781506e+09
Name: revenue, dtype: float64

Budget-Revenue-Profit

In [287]:
plt.title('financials of movies over the years')
# plt.legend(loc='upper left')
df.groupby(['release_year'])['budget'].mean().plot(figsize=(15,6), color='red', legend='Budget')
df.groupby(['release_year'])['revenue'].mean().plot(figsize=(15,6), color='blue', legend='Revenue')
df.groupby(['release_year'])['profit'].mean().plot(figsize=(15,6),color='green', legend='Profit');

Movies having budget and revenue data missing

In [290]:
len(df.query('revenue == 0 and budget == 0')), len(df.query('revenue != 0 or budget != 0'))
Out[290]:
(4672, 6163)
In [291]:
len(df.query('revenue != 0 and budget != 0')), len(df.query('revenue == 0 or budget == 0'))
Out[291]:
(3855, 6980)

We have substantial number of movie not having budget and revenue.

We can still take average of existing data of these feature and update them but the problem is that would affect our analysis because we are looking at data from 1960 to 2015 which has seen budget and revenue changed over years.

In [292]:
df.hist(figsize=(10,8));
In [293]:
df.query('revenue != 0 and budget != 0').hist(figsize=(10,8));

Let's fill in missing values with average and then check the budget, revenue and profit plot

Since fillna only works for NaN values, first replace all 0 budget and 0 revenue with Nan and then apply fillna

In [294]:
df['budget'] = df['budget'].apply(lambda x: np.NaN if x==0 else x)
In [295]:
df['revenue'] = df['revenue'].apply(lambda x: np.NaN if x==0 else x)
In [296]:
df.isnull().sum()
Out[296]:
imdb_id                    0
budget                  5667
revenue                 5985
original_title             0
cast                      75
director                  39
runtime                    0
genres                     0
production_companies    1012
release_date               0
vote_average               0
release_year               0
profit                     0
release_month              0
dtype: int64

now we can see there are 5667 movies with NaN budget and 5985 movies with NaN revenue

In [297]:
df['budget'].fillna(df['budget'].mean(), inplace = True)
In [298]:
df['revenue'].fillna(df['revenue'].mean(), inplace = True)

Calculating profit again since we have updated budget and revenue with average value

In [299]:
df['profit'] = df.apply(lambda x: x['revenue']-x['budget'], axis=1)
In [300]:
plt.title('financials of movies over the years')
# plt.legend(loc='upper left')
df.groupby(['release_year'])['budget'].mean().plot(figsize=(15,6), color='red', legend='Budget')
df.groupby(['release_year'])['revenue'].mean().plot(figsize=(15,6), color='blue', legend='Revenue')
df.groupby(['release_year'])['profit'].mean().plot(figsize=(15,6),color='green', legend='Profit');
In [301]:
# Check distribution of ratings
df['vote_average'].describe()
Out[301]:
count    10835.000000
mean         5.973069
std          0.933835
min          1.500000
25%          5.400000
50%          6.000000
75%          6.600000
max          9.200000
Name: vote_average, dtype: float64

Unique Genres

In [302]:
df.genres.value_counts()
Out[302]:
Drama                                           712
Comedy                                          711
Documentary                                     312
Drama|Romance                                   289
Comedy|Drama                                    280
Comedy|Romance                                  268
Horror|Thriller                                 259
Horror                                          253
Comedy|Drama|Romance                            222
Drama|Thriller                                  138
Comedy|Family                                   102
Action|Thriller                                 101
Thriller                                         93
Drama|Comedy                                     92
Animation|Family                                 90
Crime|Drama|Thriller                             81
Crime|Drama                                      74
Comedy|Horror                                    72
Drama|Comedy|Romance                             64
Action                                           63
Action|Comedy                                    62
Drama|History                                    58
Action|Crime|Drama|Thriller                      54
Drama|Horror|Thriller                            53
Horror|Science Fiction                           52
Action|Crime|Thriller                            52
Horror|Mystery|Thriller                          51
Comedy|Crime                                     50
Drama|Music                                      49
Documentary|Music                                49
                                               ... 
Romance|Drama|History                             1
Drama|Horror|Action|Thriller|Mystery              1
Science Fiction|Adventure|Drama|Fantasy           1
Action|Comedy|Drama|Family|Thriller               1
Crime|Action|Science Fiction                      1
Action|Adventure|Comedy|Drama|Romance             1
Thriller|Adventure|Action|Comedy|Drama            1
Music|Animation|Comedy|Family                     1
Action|Crime|Adventure                            1
Drama|Action|Music|Romance                        1
War|Drama|Mystery|Romance                         1
Fantasy|Science Fiction|Family                    1
Action|Animation|Fantasy|Science Fiction          1
Drama|Thriller|TV Movie                           1
Fantasy|Drama|Action|Comedy|Crime                 1
Action|Drama|Horror|Mystery|Thriller              1
Crime|Action|Comedy                               1
TV Movie|Fantasy|Comedy|Romance|Family            1
War|History|Action|Adventure|Drama                1
Comedy|Family|Drama|Fantasy                       1
Fantasy|Action|Adventure|Comedy                   1
Drama|Fantasy|Romance|Science Fiction             1
Romance|War|Documentary|Drama                     1
TV Movie|Comedy|Drama|Romance                     1
Action|Animation|Adventure                        1
Animation|Fantasy|Mystery                         1
Horror|Thriller|War                               1
Thriller|Action|Horror|Science Fiction|Crime      1
Thriller|Action|Mystery                           1
Thriller|Crime|Science Fiction                    1
Name: genres, Length: 2037, dtype: int64

Problem with genres is that we can't do any analysis involving it unless we split genres and create more rows for each movies having more genres seperated by pipe character

How to split each movie dataset which have multiple genres:

  1. loop through all rows
  2. select genres value and split by pipe character
  3. now create new dataframe and append that to separate dataframe called spread_df
In [304]:
spread_df = pd.DataFrame()

for index in range(df.shape[0]):
    original = df.iloc[index]
    splited = original['genres'].split('|')
    for s in splited:
        temp_df = original.copy()
        temp_df['genres'] = s
        spread_df = spread_df.append(temp_df, ignore_index = True)
        
spread_df.shape
Out[304]:
(26937, 14)
In [305]:
spread_df.head(1)
Out[305]:
budget cast director genres imdb_id original_title production_companies profit release_date release_month release_year revenue runtime vote_average
0 150000000.0 Chris Pratt|Bryce Dallas Howard|Irrfan Khan|Vi... Colin Trevorrow Action tt0369610 Jurassic World Universal Studios|Amblin Entertainment|Legenda... 1.363529e+09 2015-06-09 6.0 2015.0 1.513529e+09 124.0 6.5

we ended up with only 20 genres as a result of above operation

In [306]:
spread_df['genres'].nunique()
Out[306]:
20

check whether spreading of genres worked by grouping them and compare with original shape

In [307]:
spread_df.groupby(['imdb_id'])['genres'].sum()
Out[307]:
imdb_id
tt0035423       ComedyFantasyRomanceScience Fiction
tt0052646                     HorrorScience Fiction
tt0053559                                    Horror
tt0053580        ActionAdventureDramaHistoryWestern
tt0053604                        ComedyDramaRomance
tt0053644                              ComedyFamily
tt0053677                                    Horror
tt0053699                                    Comedy
tt0053716                             ComedyRomance
tt0053719                                    Horror
tt0053729                             ActionWestern
tt0053793                                     Drama
tt0053804                     ActionDramaHistoryWar
tt0053825                 ActionDramaRomanceWestern
tt0053877                        ComedyDramaRomance
tt0053925                            HorrorThriller
tt0053946                              DramaHistory
tt0054022                             ComedyRomance
tt0054033                                    Comedy
tt0054038           AdventureFantasyScience Fiction
tt0054047                    ActionAdventureWestern
tt0054084                                  Thriller
tt0054135                  ThrillerMusicComedyCrime
tt0054167                            HorrorThriller
tt0054195                               DramaFamily
tt0054215                       DramaHorrorThriller
tt0054269                                     Drama
tt0054292                              CrimeWestern
tt0054310              ActionDramaForeignHistoryWar
tt0054331                        ActionDramaHistory
                              ...                  
tt4856322            ActionAnimationScience Fiction
tt4897822                               Documentary
tt4900018                          MusicDocumentary
tt4902012                               Documentary
tt4908644                               Documentary
tt4909348                                    Comedy
tt4920274    ComedyTV MovieAnimationScience Fiction
tt4935334      FantasyThrillerHorrorScience Fiction
tt4938374            AnimationComedyFamilyAdventure
tt4938416            ActionAdventureAnimationFamily
tt4938602                   FamilyTV MovieAnimation
tt4941804                           AnimationFamily
tt4955162                      AnimationMusicFamily
tt4973112                               Documentary
tt4974584                               DramaHorror
tt4995590                          MusicDocumentary
tt5052966                                    Horror
tt5065822                               Documentary
tt5069564                                    Comedy
tt5083702                       ThrillerDocumentary
tt5113926                                    Horror
tt5133572                                  TV Movie
tt5133810                            FamilyTV Movie
tt5184298      Science FictionComedyAnimationFamily
tt5204384                             DramaTV Movie
tt5210380                           RomanceTV Movie
tt5223342                                 Animation
tt5227516                                     Drama
tt5297750                          DocumentaryMusic
tt6019206                               CrimeAction
Name: genres, Length: 10834, dtype: object
In [308]:
df.shape
Out[308]:
(10835, 14)

above shape comparison tells us that our spread worked.

Now we can check how many movies are released under each genre

In [309]:
spread_df.genres.value_counts()
Out[309]:
Drama              4759
Comedy             3792
Thriller           2908
Action             2380
Romance            1712
Horror             1637
Adventure          1469
Crime              1355
Family             1230
Science Fiction    1225
Fantasy             913
Mystery             810
Animation           697
Documentary         519
Music               407
History             334
War                 270
Foreign             188
TV Movie            167
Western             165
Name: genres, dtype: int64

Plot the genre for visual understanding

In [310]:
spread_df.genres.value_counts().plot(kind='bar', figsize=(10,7));

# Few other variants
# spread_df.genres.value_counts().plot(kind='bar', figsize=(10,7)).invert_xaxis();
# spread_df.genres.value_counts().plot(kind='bar', figsize=(10,7)).invert_yaxis();

vote average among genres

In [311]:
spread_df.groupby(['genres'])['vote_average'].mean().plot(kind='bar');

The above plot doesn't take into account for years, lets do the same analysis over years

What movies people rate high?

In [312]:
# spread_df.groupby(['release_year', 'genres'])['vote_average'].mean()
In [313]:
# plt.figure()
spread_df.groupby(['release_year', 'genres'])['vote_average'].mean().unstack().plot(figsize=(15,8));

let's break down above plot by just comparing two genres

In [314]:
spread_df.query('genres == "Adventure" or genres == "Horror"').groupby(['release_year', 'genres'])['vote_average'].mean().unstack().plot(figsize=(15,7));

looks like people started rating low for horror movies after 1987 and while adventure movies almost maintained their rating.

Let's see how movies are invested for same genre movies over the years

In [317]:
spread_df.query('genres == "Adventure" or genres == "Horror"').groupby(['release_year', 'genres'])['budget'].mean().unstack().plot(figsize=(15,5));

budget for Adventure movies increased after 1997 when compared to Horror

In [318]:
df.groupby(['release_month'])['profit'].mean().plot(kind='bar', figsize=(15,7));

movies released in June made more profit

Let's look at data for 2015

Comparing revenue for same genre movies over the years

In [319]:
spread_df.query('genres == "Adventure" or genres == "Horror"').groupby(['release_year', 'genres'])['revenue'].mean().unstack().plot(figsize=(15,6));

looks like Adventure movies revenue is higher as well

Budget and Revenue among genres

In [320]:
spread_df.query('release_year == 2015').groupby(['genres'])['budget', 'revenue'].mean()
Out[320]:
budget revenue
genres
Action 4.951033e+07 1.777273e+08
Adventure 6.517880e+07 2.362795e+08
Animation 4.427656e+07 1.638711e+08
Comedy 3.230128e+07 9.917246e+07
Crime 4.057312e+07 1.305806e+08
Documentary 2.968253e+07 7.704486e+07
Drama 2.684914e+07 7.709830e+07
Family 5.086454e+07 1.706213e+08
Fantasy 5.312588e+07 1.715952e+08
History 2.615386e+07 7.157951e+07
Horror 2.621983e+07 8.126920e+07
Music 2.574183e+07 7.813088e+07
Mystery 3.195320e+07 6.971523e+07
Romance 2.697277e+07 8.008765e+07
Science Fiction 4.801471e+07 1.729722e+08
TV Movie 2.936376e+07 8.922066e+07
Thriller 3.184324e+07 1.032056e+08
War 3.744503e+07 1.081953e+08
Western 4.709189e+07 1.594336e+08
In [321]:
spread_df.query('release_year == 2015').groupby(['genres'])['budget'].mean().plot(kind='bar', alpha=0.5, color='red', figsize=(15,7))
spread_df.query('release_year == 2015').groupby(['genres'])['revenue'].mean().plot(kind='bar', alpha=0.5, color='blue', figsize=(15,7))
plt.legend();

Profit among genres

In [322]:
spread_df.query('release_year == 2015').groupby(['genres'])['profit'].mean().plot(kind='bar', figsize=(15,7));

Adventure, Action, Animation, Fantasy, Science Fiction, Family movies are making good profit

In [323]:
df.query('release_year == 2015').groupby(['release_month'])['profit'].mean().plot(kind='bar', figsize=(15,7));

in 2015, December released movies made more profit just ahead of June which has overall lead when compared over the years.

Runtime distribution

In [324]:
df.runtime.describe()
Out[324]:
count    10835.000000
mean       102.161790
std         31.263769
min          0.000000
25%         90.000000
50%         99.000000
75%        111.000000
max        900.000000
Name: runtime, dtype: float64

have to remove outliers

Better way to drop is by applying condition and create new dataframe

In [327]:
runtime_df = df[df.runtime < 350]
In [328]:
plt.title('Runtime Distribution of all movies')
plt.xlabel('minutes')
runtime_df['runtime'].plot(kind='hist', figsize=(10, 6), bins=30);

most movie runtime is around 90 to 110 minutes.

What month movies are released most

In [329]:
df['release_month'].value_counts()
Out[329]:
9     1328
10    1147
12     981
8      915
1      912
6      826
3      822
11     814
5      808
7      798
4      797
2      687
Name: release_month, dtype: int64

Plotting the above

In [330]:
plt.figure()
df['release_month'].value_counts().sort_index().plot(kind='bar', figsize=(12, 6));
Months = ['January', 'February', 'March', 'April', 'May', 'June', 'July', 'August', 'September', 'October', 'November', 'December']
plt.title('# of movies released by month')
plt.xlabel(Months, fontsize=12)
plt.show()

Conclusions

So looking at all the investigation done so far, if one were to make a movie I can recommend following:

  • Choose from following production company
    • Paramount Pictures
    • Universal Pictures
    • Warner Bros
    • Walt Disney Pictures
    • Metro-Goldwyn-Mayer
    • Columbia Pictures
    • New Line Cinema
    • Touchstone Pictures
    • 20th Century Fox
  • Choose from following genre
    • Adventure
    • Action
    • Animation
    • Fantasy
    • Science Fiction
    • Family
  • Should make movie around 90-110 minutes runtime
  • June and December have better chances of being successful
  • June has lesser movies released but those movies also made more profit