hands-on-data-analysis 第二单元 2,3节

简介: 数据合并——concat横向合并

hands-on-data-analysis 第二单元 2,3节

第二节 数据重构

万事开头记得导入基本的库:

# 导入基本库
import numpy as np
import pandas as pd

2.1.数据合并——concat横向合并

官方文档:

pandas.concat — pandas 1.4.2 documentation (pydata.org)

pandas中DataFrame数据合并连接(merge、join、concat

text_left_up,text_right_up两张表,如果横向合并为一张表(就是列与列拼接在一起)

text_left_up

PassengerId Survived Pclass Name
0 1 0 3 Braund, Mr. Owen Harris
1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th...
2 3 1 3 Heikkinen, Miss. Laina
3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel)
4 5 0 3 Allen, Mr. William Henry

text_right_up:

Sex Age SibSp Parch Ticket Fare Cabin Embarked
0 male 22.0 1.0 0.0 A/5 21171 7.2500 NaN S
1 female 38.0 1.0 0.0 PC 17599 71.2833 C85 C
2 female 26.0 0.0 0.0 STON/O2. 3101282 7.9250 NaN S
3 female 35.0 1.0 0.0 113803 53.1000 C123 S
4 male 35.0 0.0 0.0 373450 8.0500 NaN S
list_up = [text_left_up,text_right_up]
result_up = pd.concat(list_up,axis=1)
result_up.head()

得到:

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked
0 1.0 0.0 3.0 Braund, Mr. Owen Harris male 22.0 1.0 0.0 A/5 21171 7.2500 NaN S
1 2.0 1.0 1.0 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1.0 0.0 PC 17599 71.2833 C85 C
2 3.0 1.0 3.0 Heikkinen, Miss. Laina female 26.0 0.0 0.0 STON/O2. 3101282 7.9250 NaN S
3 4.0 1.0 1.0 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1.0 0.0 113803 53.1000 C123 S
4 5.0 0.0 3.0 Allen, Mr. William Henry male 35.0 0.0 0.0 373450 8.0500 NaN S

就好比,把带有小明的学号的表和带有小明成绩的表合在一起。

2.2.数据合并——concat纵向合并

官方文档:

pandas.concat — pandas 1.4.2 documentation (pydata.org)

pandas中DataFrame数据合并连接(merge、join、concat

将train-left-down和train-right-down横向合并为一张表,并保存这张表为result_down。然后将上边的result_up和result_down纵向合并为result。

text_left_down的数据为:

PassengerId Survived Pclass Name
0 440 0 2 Kvillner, Mr. Johan Henrik Johannesson
1 441 1 2 Hart, Mrs. Benjamin (Esther Ada Bloomfield)
2 442 0 3 Hampe, Mr. Leon
3 443 0 3 Petterson, Mr. Johan Emil
4 444 1 2 Reynaldo, Ms. Encarnacion

text_right_down数据为:

Sex Age SibSp Parch Ticket Fare Cabin Embarked
0 male 31.0 0 0 C.A. 18723 10.500 NaN S
1 female 45.0 1 1 F.C.C. 13529 26.250 NaN S
2 male 20.0 0 0 345769 9.500 NaN S
3 male 25.0 1 0 347076 7.775 NaN S
4 female 28.0 0 0 230434 13.000 NaN S
list_down=[text_left_down,text_right_down]
result_down = pd.concat(list_down,axis=1)
result = pd.concat([result_up,result_down])
result.head()

合并后的表为:

PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked
0 1.0 0.0 3.0 Braund, Mr. Owen Harris male 22.0 1.0 0.0 A/5 21171 7.2500 NaN S
1 2.0 1.0 1.0 Cumings, Mrs. John Bradley (Florence Briggs Th... female 38.0 1.0 0.0 PC 17599 71.2833 C85 C
2 3.0 1.0 3.0 Heikkinen, Miss. Laina female 26.0 0.0 0.0 STON/O2. 3101282 7.9250 NaN S
3 4.0 1.0 1.0 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1.0 0.0 113803 53.1000 C123 S
4 5.0 0.0 3.0 Allen, Mr. William Henry male 35.0 0.0 0.0 373450 8.0500 NaN S

2.3.数据合并——join

官方文档:
pandas.DataFrame.join — pandas 1.4.2 documentation (pydata.org)

Join columns of another DataFrame.

Join columns with other DataFrame either on index or on a key column. Efficiently join multiple DataFrame objects by index at once by passing a list.

从官方文档上可以知道,join的方式比较灵活。

可以在 索引 上将 列 与其他 DataFrame 连接。 也可以通过传递一个列表,一次有效地按索引连接多个 DataFrame 对象。

参数有:

DataFrame.join(other, on=None, how='left', lsuffix='', rsuffix='', sort=False)

2.4. concat 与 join 比较

concat、join等的比较

Merge, join, concatenate and compare — pandas 1.4.2 documentation (pydata.org)

第三节 GroupBy 接口

官方文档:

pandas.DataFrame.groupby — pandas 1.4.2 documentation (pydata.org)

目录
相关文章
|
1月前
|
机器学习/深度学习 测试技术 算法
文献解读-DNAscope: High accuracy small variant calling using machine learning
在这项研究中,研究组证明了DNAscope在不同样本和不同覆盖度水平下都能达到比DNAseq更高的准确性。使用GA4GH分层区域进行的分层分析,能够确认DNAscope在大多数分层区域中都具有高准确性,并突显了DNAscope在插入缺失(indels)和包含变异检测较困难的基因组区域的分层中具有更高的准确性。DNAscope结合了GATK's HaplotypeCaller中使用的成熟数学和统计模型,以及用于变异基因型分析的机器学习方法,在保持计算效率的同时实现了卓越的准确性。
29 3
文献解读-DNAscope: High accuracy small variant calling using machine learning
|
16天前
|
算法 安全 数据挖掘
文献解读-Transcriptional Start Site Coverage Analysis in Plasma Cell-Free DNA Reveals Disease Severity and Tissue Specificity of COVID-19 Patients
这项研究展示了 cfDNA 分析在揭示新冠肺炎进展中的组织参与情况和疾病机制方面的潜力。它强调了 cfDNA 作为无创生物标志物在疾病严重程度检测、患者监测和预后评估中的应用价值。这种方法为理解新冠肺炎的病理生理学提供了新的视角,并可能帮助开发更有针对性的治疗策略。
25 2
|
算法 Linux Shell
SGAT丨Single Gene Analysis Tool
SGAT丨Single Gene Analysis Tool
|
机器学习/深度学习 自然语言处理 算法
Joint Information Extraction with Cross-Task and Cross-Instance High-Order Modeling 论文解读
先前的信息抽取(IE)工作通常独立地预测不同的任务和实例(例如,事件触发词、实体、角色、关系),而忽略了它们的相互作用,导致模型效率低下。
96 0
《Data infrastructure architecture for a medium size organization tips for collecting, storing and analysis》电子版地址
Data infrastructure architecture for a medium size organization: tips for collecting, storing and analysis
90 0
《Data infrastructure architecture for a medium size organization tips for collecting, storing and analysis》电子版地址
《40 Must Know Questions to test a data scientist on Dimensionality Reduction techniques》电子版地址
40 Must Know Questions to test a data scientist on Dimensionality Reduction techniques
98 0
《40 Must Know Questions to test a data scientist on Dimensionality Reduction techniques》电子版地址
《Distributed End-to-End Drug Similarity Analytics and Visualization Workflow》电子版地址
Distributed End-to-End Drug Similarity Analytics and Visualization Workflow
79 0
《Distributed End-to-End Drug Similarity Analytics and Visualization Workflow》电子版地址
《Fighting Cybercrime A Joint Task Force of Real-Time Data and Human Analytics》电子版地址
Fighting Cybercrime: A Joint Task Force of Real-Time Data and Human Analytics
86 0
《Fighting Cybercrime A Joint Task Force of Real-Time Data and Human Analytics》电子版地址
《Audio Tagging with Compact Feedforward Sequential Memory Network and Audio-to-Audio Ratio Based Data Augmentation》电子版地址
Audio Tagging with Compact Feedforward Sequential Memory Network and Audio-to-Audio Ratio Based Data Augmentation
83 0
《Audio Tagging with Compact Feedforward Sequential Memory Network and Audio-to-Audio Ratio Based Data Augmentation》电子版地址
|
机器学习/深度学习 算法 数据挖掘
Re18:读论文 GCI Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis
Re18:读论文 GCI Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis
Re18:读论文 GCI Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis