<jupyter_start><jupyter_text>TaskOptimise team picks for footballsurvivor.co.uk competiton. Game rules: pick a Premier League team each week that you expect to win their match. Start the season with 10 lives :* If team wins: drop 0 lives* If team draws: drop 1 life* If team loses: drop 2 lives**You can only pick each team once during the course of the season**If you run out of lives, you lose. If you survive, the winner is the person with the most remaining lives. ApproachApply linear sum assignment (a.k.a Hungarian algorithm) to select n teams over k weeks, such that a single team is assigned to exactly one week.The assignment will be made based on maximizing some value. In this case we will use a team's weekly win probability calculated by fivethirtyeight.com Standalone python script: `survivor_league_optimise.py`<jupyter_code>import json 
import survivor_league_optimise as optimise

%matplotlib inline
%load_ext autoreload
%autoreload 2<jupyter_output><empty_output><jupyter_text>[1] fivethirtyeight.com dataThe first step is to download and parse data from fivethirtyeight.com (which we will refer to as **fte**) to get a modelled probability of a team winning a match against a specific opponent, based on their recent performances (and other factors). The sum of these probabilities will be the value we are trying to maximize when we make our team picks.The **fte** data in nicely packaged in a json api which we will download and parse into a dataframe. ![title](fivethirtyeight.PNG)<jupyter_code>df_fte = optimise.get_fte_data(year=2017)
df_fte.head()<jupyter_output><empty_output><jupyter_text>[2] footballsurvivor.co.uk dataNow that we have the probability of a team winning a particular match, we need to know which "gameweek"   that match belongs to, as our aim is to pick a single team for each gameweek. A gameweek is defined as a round of matches in which every team plays exactly once. To get this data we need to scrape footballsurvivor.co.uk (**fs**), which is slightly more convoluted as we first need to login to the website, whilst the data itself is embedded in the html.We create a `requests` session object to persist our authentication parameters after logging in, then use xpath to extract the gameweek definitions from the html.We will also return a boolean indicating whether a team was picked in that gameweek. At the start of the season this will be False for all records, but will be useful later on if we want to recalculate our picks, as we can easily exclude teams which have already been picked.![fs2](footballsurvivor.PNG)<jupyter_code>with open('fs_credentials.json', 'rb') as fs_cred_file:
    fs_credentials = json.load(fs_cred_file)

# login to site
fs_session = optimise.fs_session_login(fs_credentials)

# get/parse html
fs_league_url = 'https://footballsurvivor.co.uk/leagues/geo_punters_winner_takes_all/entries/70392/fixtures'
df_fs = optimise.get_fs_data(fs_session, fs_league_url)
df_fs.head()<jupyter_output><empty_output><jupyter_text>[3] Merging dataframes We now have a dataframe for each data source, so the next step is to merge them into a single dataframe. Whilst each dataframe has the same record unique "key" ('team', 'opp', 'loc'), the format of the team names for **fte** data is different to **fs**.We will firstly map the **fs** team names to their **fte** equivalents (as they are less verbose), then merge the dataframes.<jupyter_code>df_merged = optimise.merge_fs_fte_data(df_fs, df_fte) 
df_merged.head()<jupyter_output><empty_output><jupyter_text>[4] Reshape to probability matrix Currently we have a long-form dataframe with one row per team/gameweek combination. We need to reshape it into a wide "matrix" with one row per gameweek and one column per team as this is the format required by the the `scipy` function we use to optimise our picks. The values of the matrix will be the win probability (p_win) of that team/gameweek combination.<jupyter_code>df_prob_matrix = optimise.get_probability_matrix(df_merged)
df_prob_matrix.iloc[:5,:5]<jupyter_output><empty_output><jupyter_text>[5] Optimise picksThe final step is to apply `scipy.linear_sum_assigment` to our matrix which selects exactly one element from each row and each column, such that the sum of the selected elements are maximized.To recap, the footballsurvivor competition requires us to pick a single team for each gamweek, without repeating any team. As better teams will consistently have a higher win probability across all gameweeks, one approach could be to pick teams in order of probability, with the most likely winners picked first. However, once we have exhausted the all stronger picks, we will have fewer gameweeks left in which to try and pick a winning match for the weaker teams.The motivation for our chosen approach is that by trying to maximize the probability of winning for all teams, we will (on average) have a better chance of picking a winner each time.<jupyter_code>df_picks = optimise.optimise_picks(df_prob_matrix, value_label='p_win')
df_picks.head()<jupyter_output><empty_output><jupyter_text>We now have out optimised picks!Plotting the entire probability matrix, with our selected picks highlighted<jupyter_code>optimise.plot_picks_heatmap(df_prob_matrix, df_picks, plot_size=(11,7))<jupyter_output><empty_output><jupyter_text>Notice that some teams have not been picked at all, this is because the footballsurvivor league used in this example was started towards the end of the season, and so their are not enough gameweeks left to require all team to be picked.This is not a problem as the algorithm automatically handles a non-square matrix by only picking the teams which maximize the objective (sum of probabilities). Calculating the average probability for our optimised picks:<jupyter_code>df_picks.p_win.mean()<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup