數據的分組匯總-基於R的reshape包

01-28

作者：劉順祥公眾號：每天進步一點點2015 (微信ID：lsxxx2011)
配套教程：手把手教你做文本挖掘 https://edu.hellobi.com/course/181

reshape包中的函數提供了解決匯總問題的統一辦法，該包的核心思想是創造一個「熔化」的數據集版本(通過melt函數)，然後將其投入(cast函數)到一個所希望的目標對象中。

通過melt函數「熔化」一個數據框、列表或數組，首先需要將變數分成編號變數和分析變數。默認情況下，該函數將因子和整數值變數設為編號變數，其餘變數為分析變數。

metl函數語法：

1）熔化一個數據框：id.vars指定編號變數，measure.vars指定分析變數

melt(data, id.vars, measure.vars,

variable_name = "variable", na.rm = !preserve.na, preserve.na = TRUE, ...)

2）熔化一個數組

melt(data, varnames = names(dimnames(data)), ...)

3）熔化一個列表

melt(data, ..., level=1)

應用：本文所採用的數據為R自帶的數據集state.x77、iris及隨機數生成的數據框

#生成美國50個州的人口、收入的數據nnstates <- data.frame(state = row.names(state.x77),region = state.region,nstate.x77,row.names = 1:50)n#查看數據集nn> head(states)nstate region Population Income Illiteracy Life.Exp Murder HS.Grad Frost Arean1 Alabama South 3615 3624 2.1 69.05 15.1 41.3 20 50708n2 Alaska West 365 6315 1.5 69.31 11.3 66.7 152 566432n3 Arizona West 2212 4530 1.8 70.55 7.8 58.1 15 113417n4 Arkansas South 2110 3378 1.9 70.66 10.1 39.9 65 51945n5 California West 21198 5114 1.1 71.71 10.3 62.6 20 156361n6 Colorado West 2541 4884 0.7 72.06 6.8 63.9 166 103766n#採用melt函數熔化數據框(states)nm_states <- melt(states)n#查看熔化後的數據結構nn> head(m_states) #在不指定編號變數時，melt會顯示被自動轉為編號變數的變數名稱nstate region variable valuen1 Alabama South Population 3615n2 Alaska West Population 365n3 Arizona West Population 2212n4 Arkansas South Population 2110n5 California West Population 21198n6 Colorado West Population 2541n#可以通過id.vars和measure.vars參數指定感興趣的分析變數和分組變數nn> head(melt(states,id.vars = state,measure.vars = Income))nstate variable valuen1 Alabama Income 3624n2 Alaska Income 6315n3 Arizona Income 4530n4 Arkansas Income 3378n5 California Income 5114n6 Colorado Income 4884n#我們發現上面的「熔化」數據，除了指定的或默認的id變數，還會額外產生variable變數和value變數，這兩個變數分別存放感興趣的分析變數名稱和實際的數值n

通過melt函數，將數據框「熔化」後放入cast函數進行統計匯總。cast函數語法如下：

cast(data, formula = ... ~ variable, fun.aggregate=NULL, ...,nmargins=FALSE, subset=TRUE, df=FALSE, fill=NULL, add.missing=FALSE,nvalue = guess_value(data))n

其中data為一個「熔化」後的對象；formula為顯式公式，公式左邊代表輸出結果的行變數，右邊則代表輸出結果的列變數；fun.aggregate為匯總函數，默認情況下使用length,最關鍵的是該參數可以指定一個自編函數。

應用：計算按地區分組的每個變數的均值

reg_mean1 <- cast(m_states,region~variable,mean)nreg_mean2 <- cast(m_states,variable~region,mean)n#查看以上兩個結果，並比較formula參數顛倒的差異(前者以地區作為結果的行變數，後者以興趣變數的名稱作為行變數)n> head(reg_mean1)nregion Population Income Illiteracy Life.Exp Murder HS.Grad Frost Arean1 Northeast 5495.111 4570.222 1.000000 71.26444 4.722222 53.96667 132.7778 18141.00n2 South 4208.125 4011.938 1.737500 69.70625 10.581250 44.34375 64.6250 54605.12n3 North Central 4803.000 4611.083 0.700000 71.76667 5.275000 54.51667 138.8333 62652.00n4 West 2915.308 4702.615 1.023077 71.23462 7.215385 62.00000 102.1538 134463.00n> head(reg_mean2)nvariable Northeast South North Central Westn1 Population 5495.111111 4208.12500 4803.00000 2915.307692n2 Income 4570.222222 4011.93750 4611.08333 4702.615385n3 Illiteracy 1.000000 1.73750 0.70000 1.023077n4 Life.Exp 71.264444 69.70625 71.76667 71.234615n5 Murder 4.722222 10.58125 5.27500 7.215385n6 HS.Grad 53.966667 44.34375 54.51667 62.000000n#接下來，創建一個自編函數應用到cast函數中，本次使用到的數據集為irisnfun <- function(x) {nrequire(fBasics)nn = sum(!is.na(x))nnmiss = sum(is.na(x))nm = mean(x,na.rm = TRUE)ns = sd(x,na.rm = TRUE)nmax = max(x,na.rm = TRUE)nmin = min(x,na.rm = TRUE)nrange = max-minnskew = skewness(x,na.rm = TRUE)nkurto = kurtosis(x,na.rm = TRUE)nnreturn(c(n = n,nmiss = nmiss,mean = m,sd = s,max = max,min = min,range = range,skewness = skew,kurtosis = kurto))n}n#數據「熔化」與匯總nm_iris <- melt(iris)n#為了方便版面顯示，這裡將輸出結果設置為列表格式(注意，我在variable前面加了.|)nsummary_result <- cast(m_iris,variable~.|Species,fun)n> summary_resultn$setosanvariable n nmiss mean sd max min range skewness kurtosisn1 Sepal.Length 50 0 5.006 0.3524897 5.8 4.3 1.5 0.11297784 -0.4508724n2 Sepal.Width 50 0 3.428 0.3790644 4.4 2.3 2.1 0.03872946 0.5959507n3 Petal.Length 50 0 1.462 0.1736640 1.9 1.0 0.9 0.10009538 0.6539303n4 Petal.Width 50 0 0.246 0.1053856 0.6 0.1 0.5 1.17963278 1.2587179n$versicolornvariable n nmiss mean sd max min range skewness kurtosisn1 Sepal.Length 50 0 5.936 0.5161711 7.0 4.9 2.1 0.09913926 -0.6939138n2 Sepal.Width 50 0 2.770 0.3137983 3.4 2.0 1.4 -0.34136443 -0.5493203n3 Petal.Length 50 0 4.260 0.4699110 5.1 3.0 2.1 -0.57060243 -0.1902555n4 Petal.Width 50 0 1.326 0.1977527 1.8 1.0 0.8 -0.02933377 -0.5873144n$virginicanvariable n nmiss mean sd max min range skewness kurtosisn1 Sepal.Length 50 0 6.588 0.6358796 7.9 4.9 3.0 0.1110286 -0.2032597n2 Sepal.Width 50 0 2.974 0.3224966 3.8 2.2 1.6 0.3442849 0.3803832n3 Petal.Length 50 0 5.552 0.5518947 6.9 4.5 2.4 0.5169175 -0.3651161n4 Petal.Width 50 0 2.026 0.2746501 2.5 1.4 1.1 -0.1218119 -0.7539586n

如果想選擇特定的分析變數，可以通過subset參數實現,一般與%in%聯合使用。a%in%b 表示a的元素是否為b的子集

#分析Sepal.Length變數的匯總信息nsl_summary <- cast(m_iris,Species~variable,fun,subset = variable %in% Sepal.Length)n> sl_summarynSpecies Sepal.Length_n Sepal.Length_nmiss Sepal.Length_mean Sepal.Length_sd Sepal.Length_max Sepal.Length_minn1 setosa 50 0 5.006 0.3524897 5.8 4.3n2 versicolor 50 0 5.936 0.5161711 7.0 4.9n3 virginica 50 0 6.588 0.6358796 7.9 4.9nSepal.Length_range Sepal.Length_skewness Sepal.Length_kurtosisn1 1.5 0.11297784 -0.4508724n2 2.1 0.09913926 -0.6939138n3 3.0 0.11102862 -0.2032597n

對於多個變數的分組統計，cast函數中顯式公式的左邊或右邊用+連接多個分組變數

應用：使用隨機數函數生成泊松分布的離散變數

#模擬數據的生成ndata <- data.frame(x = rpois(100,2),y = rpois(100,3),z = runif(100,10,20))n> head(data)nx y zn1 1 2 14.48573n2 2 3 16.50317n3 2 2 19.27258n4 1 4 18.86011n5 2 4 15.97173n6 3 4 10.88769n#數據「熔化」nnm_data <- melt(data,measure.vars = z)n> head(m_data)nx y variable valuen1 1 2 z 14.48573n2 2 3 z 16.50317n3 2 2 z 19.27258n4 1 4 z 18.86011n5 2 4 z 15.97173n6 3 4 z 10.88769n#多變數分組統計nsummary_data <- cast(m_data,x+y~variable,c(mean,min,max,median))n> head(summary_data)nx y z_mean z_min z_max z_mediann1 0 0 17.90826 17.90826 17.90826 17.90826n2 0 2 17.68077 17.68077 17.68077 17.68077n3 0 3 14.43726 12.43668 16.96619 13.90891n4 0 4 16.54500 13.90112 19.18888 16.54500n5 0 5 12.87987 12.87987 12.87987 12.87987n6 0 6 16.51866 16.51866 16.51866 16.51866n#需要注意的是，表達式左邊的最後一個變數是變化最快的n最後再介紹cast函數中顯式公式的幾種變形：n#以y的每一個值單獨成一個列，統計x分組下的匯總值nreshape1 <- cast(m_data,x~y+variable,mean)n> reshape1nx 0_z 1_z 2_z 3_z 4_z 5_z 6_z 7_zn1 0 17.90826 NaN 17.68077 14.43726 16.54500 12.87987 16.51866 NaNn2 1 15.66487 15.67371 15.74976 14.99201 15.58618 17.01104 11.25401 15.05334n3 2 NaN 14.34325 14.31375 15.82231 16.09901 15.45799 NaN NaNn4 3 13.60504 16.51666 16.03237 17.17504 15.41220 11.25861 11.23605 NaNn5 4 16.86313 14.29701 12.59724 NaN 10.60965 14.90316 NaN NaNn6 5 NaN 18.48920 19.61777 NaN NaN NaN NaN 14.27881n7 7 15.55771 NaN NaN NaN NaN NaN NaN NaNn8 8 NaN NaN 12.95656 NaN NaN NaN NaN NaNn#這裡的NaN表示x和y的組合下沒有z的值n#用豎線(|)隔開variable和y,返回列表形式，y值為列表的元素，元素內容又以x分組統計nreshape2 <- cast(m_data,x~variable|y,mean)n> reshape2n$`0`nx zn1 0 17.90826n2 1 15.66487n3 3 13.60504n4 4 16.86313n5 7 15.55771n$`1`nx zn1 1 15.67371n2 2 14.34325n3 3 16.51666n4 4 14.29701n5 5 18.48920n...n...n...n