1609885620

Talk

With larger datasets we need to be smarter about how we use Pandas to get results. We’ll look at strategies to shrink our data to get more into RAM, offload computation to tools like Dask or Vaex, store with Parquet or SQLite, make calculations faster and retain debuggability.

Speaker

Ian is a Chief Data Scientist and Coach, he helps co-organise the annual PyDataLondon conference with 700+ attendees and the associated 11,000+ member monthly meetup. He runs the established Mor Consulting Data Science consultancy in London, gives conference talks internationally often as keynote speaker and is the author of the bestselling O’Reilly book High Performance Python (2nd edition). He has 18 years of experience as a senior data science leader, trainer and team coach. For fun he’s walked by his high-energy Springer Spaniel, surfs the Cornish coast and drinks fine coffee. Past talks and articles can be found at: IanOzsvald.com

#pandas #python #data-analysis #developer

1586702221

In this post, we will learn about pandas’ data structures/objects. Pandas provide two type of data structures:-

Pandas Series is a one dimensional indexed data, which can hold datatypes like integer, string, boolean, float, python object etc. A Pandas Series can hold only one data type at a time. The axis label of the data is called the index of the series. The labels need not to be unique but must be a hashable type. The index of the series can be integer, string and even time-series data. In general, Pandas Series is nothing but a column of an excel sheet with row index being the index of the series.

Pandas dataframe is a primary data structure of pandas. Pandas dataframe is a two-dimensional size mutable array with both flexible row indices and flexible column names. In general, it is just like an excel sheet or SQL table. It can also be seen as a python’s dict-like container for series objects.

#python #python-pandas #pandas-dataframe #pandas-series #pandas-tutorial

1602550800

Pandas is used for data manipulation, analysis and cleaning.

**What are Data Frames and Series?**

**Dataframe** is a two dimensional, size mutable, potentially heterogeneous tabular data.

It contains rows and columns, arithmetic operations can be applied on both rows and columns.

**Series** is a one dimensional label array capable of holding data of any type. It can be integer, float, string, python objects etc. Panda series is nothing but a column in an excel sheet.

s = pd.Series([1,2,3,4,56,np.nan,7,8,90])

print(s)

**How to create a dataframe by passing a numpy array?**

- d= pd.date_range(‘20200809’,periods=15)
- print(d)
- df = pd.DataFrame(np.random.randn(15,4), index= d, columns = [‘A’,’B’,’C’,’D’])
- print(df)

#pandas-series #pandas #pandas-in-python #pandas-dataframe #python

1616050935

In my last post, I mentioned the groupby technique in Pandas library. After creating a groupby object, it is limited to make calculations on grouped data using groupby’s own functions. For example, in the last lesson, we were able to use a few functions such as mean or sum on the object we created with groupby. But with the aggregate () method, we can use both the functions we have written and the methods used with groupby. I will show how to work with groupby in this post.

#pandas-groupby #python-pandas #pandas #data-preprocessing #pandas-tutorial

1616395265

In my last post, I mentioned summarizing and computing descriptive statistics using the Pandas library. To work with data in Pandas, it is necessary to load the data set first. Reading the data set is one of the important stages of data analysis. In this post, I will talk about reading and writing data.

Before starting the topic, our Medium page includes posts on data science, artificial intelligence, machine learning, and deep learning. Please don’t forget to follow us on ** Medium** 🌱 to see these posts and the latest posts.

Let’s get started.

#python-pandas-tutorial #pandas-read #pandas #python-pandas

1616395265

In my last post, I mentioned summarizing and computing descriptive statistics using the Pandas library. To work with data in Pandas, it is necessary to load the data set first. Reading the data set is one of the important stages of data analysis. In this post, I will talk about reading and writing data.

Before starting the topic, our Medium page includes posts on data science, artificial intelligence, machine learning, and deep learning. Please don’t forget to follow us on ** Medium** 🌱 to see these posts and the latest posts.

Let’s get started.

#python-pandas-tutorial #pandas-read #pandas #python-pandas