Skip to content

Python crashes when executing memory_usage(deep=True) on a sparse series #19368

Closed
@quale1

Description

@quale1

Code Sample, a copy-pastable example if possible

import pandas as pd

s = pd.Series([None])
s.to_sparse().memory_usage(deep=True)

# crashes - Kernel died, restarting

Problem description

Executing the memory_usage(deep=True) method on a sparse series crashes Python. (With deep=False the method works as expected.)

Expected Output

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.3.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.22.0
pytest: 3.2.1
pip: 9.0.1
setuptools: 36.5.0.post20170921
Cython: 0.26.1
numpy: 1.13.3
scipy: 0.19.1
pyarrow: None
xarray: None
IPython: 6.1.0
sphinx: 1.6.3
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.1.0
openpyxl: 2.4.8
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.0.2
lxml: 4.1.0
bs4: 4.6.0
html5lib: 0.999999999
sqlalchemy: 1.1.13
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

Activity

jreback

jreback commented on Jan 24, 2018

@jreback
Contributor

sparse is not fully test covered. a pull request to fix is welcome!

added this to the Next Major Release milestone on Jan 24, 2018
hexgnu

hexgnu commented on Jan 29, 2018

@hexgnu
Contributor

I hooked up gdb and tracked down the issue to inside of the lib.pyx which assumes that series's are of length > 0. I made a PR that should fix it, though we'll see how the tests chooch.

modified the milestones: Next Major Release, 0.23.0 on Feb 6, 2018
added a commit that references this issue on Feb 28, 2018
e4ddbaf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

      Development

      Participants

      @hexgnu@jreback@quale1

      Issue actions

        Python crashes when executing memory_usage(deep=True) on a sparse series · Issue #19368 · pandas-dev/pandas