To study:
http://flovv.github.io/From_descritpive_to_prescriptive/
https://icyrock.com/blog/2013/12/linear-programming-in-r-using-lpsolve/
http://lpsolve.r-forge.r-project.org/
http://horicky.blogspot.co.uk/2013/01/optimization-in-r.html
http://lpsolve.sourceforge.net/5.5/R.htm
Pokazywanie postów oznaczonych etykietą R. Pokaż wszystkie posty
Pokazywanie postów oznaczonych etykietą R. Pokaż wszystkie posty
środa, 15 lutego 2017
piątek, 25 listopada 2016
Marketing with R, very interesting articles.
My search started with the question, how to reduce the huge amount of information about the history of purchases, contacts of hundred thousands of customers to manageable few variables.
I came across Recency, Frequency, Monetary value (RFM) analysis.
I came across Recency, Frequency, Monetary value (RFM) analysis.
Review articles of these users on R bloggers.
https://www.r-bloggers.com/ author/jack-han/
https://www.r-bloggers.com/ author/analyze-core-r- language/
https://www.r-bloggers.com/
https://www.r-bloggers.com/
poniedziałek, 24 października 2016
ReadingList: Data Bases in R
1. Data Table by Matt Dowle.
Cheatsheet: https://s3.amazonaws.com/assets.datacamp.com/img/blog/data+table+cheat+sheet.pdf
https://www.analyticsvidhya.com/blog/2016/05/data-table-data-frame-work-large-data-sets/
Basics: https://www.dezyre.com/data-science-in-r-programming-tutorial/r-data-table-tutorial
https://www.r-bloggers.com/a-data-table-r-tutorial-by-datacamp-intro-to-dti-j-by/
https://www.r-bloggers.com/elegant-fast-data-manipulation-with-data-table/
Adv: https://www.r-bloggers.com/advanced-tips-and-tricks-with-data-table/
https://www.r-bloggers.com/efficient-aggregation-and-more-using-data-table/
Quick union: https://www.r-bloggers.com/r-quick-tip-collapse-a-lists-of-data-frames-with-data-table/
Aggregation: https://www.r-bloggers.com/two-of-my-favorite-data-table-features/
Binning: https://www.r-bloggers.com/using-data-table-for-binning/
Joining: http://www.magesblog.com/2014/06/the-joy-of-joining-datatables.html#more
Manipulation the fast way: http://www.magesblog.com/2012/11/datatable-rocks-data-manipulation-fast.html#more
Solving common problems: https://www.r-bloggers.com/solve-common-r-problems-efficiently-with-data-table/
Matt Dowle presentations:
http://datascience.la/matt-dowles-data-table-talk-from-user-2014/
Roll:
https://www.r-bloggers.com/understanding-data-table-rolling-joins/
https://www.r-bloggers.com/advanced-tips-and-tricks-with-data-table/
All articles:
https://github.com/Rdatatable/data.table/wiki/Articles
H2O data table?
http://t.co/TYGG6KcMpx
https://www.analyticsvidhya.com/blog/2016/05/h2o-data-table-build-models-large-data-sets/
2. Dplyr by Hadley Wickham... plenty of stuff everywhere (to be elaborated on)
Cheatsheet: https://s3.amazonaws.com/assets.datacamp.com/img/blog/data+table+cheat+sheet.pdf
https://www.analyticsvidhya.com/blog/2016/05/data-table-data-frame-work-large-data-sets/
Basics: https://www.dezyre.com/data-science-in-r-programming-tutorial/r-data-table-tutorial
https://www.r-bloggers.com/a-data-table-r-tutorial-by-datacamp-intro-to-dti-j-by/
https://www.r-bloggers.com/elegant-fast-data-manipulation-with-data-table/
Adv: https://www.r-bloggers.com/advanced-tips-and-tricks-with-data-table/
https://www.r-bloggers.com/efficient-aggregation-and-more-using-data-table/
Quick union: https://www.r-bloggers.com/r-quick-tip-collapse-a-lists-of-data-frames-with-data-table/
Aggregation: https://www.r-bloggers.com/two-of-my-favorite-data-table-features/
Binning: https://www.r-bloggers.com/using-data-table-for-binning/
Joining: http://www.magesblog.com/2014/06/the-joy-of-joining-datatables.html#more
Manipulation the fast way: http://www.magesblog.com/2012/11/datatable-rocks-data-manipulation-fast.html#more
Solving common problems: https://www.r-bloggers.com/solve-common-r-problems-efficiently-with-data-table/
Matt Dowle presentations:
http://datascience.la/matt-dowles-data-table-talk-from-user-2014/
Roll:
https://www.r-bloggers.com/understanding-data-table-rolling-joins/
https://www.r-bloggers.com/advanced-tips-and-tricks-with-data-table/
All articles:
https://github.com/Rdatatable/data.table/wiki/Articles
H2O data table?
http://t.co/TYGG6KcMpx
https://www.analyticsvidhya.com/blog/2016/05/h2o-data-table-build-models-large-data-sets/
2. Dplyr by Hadley Wickham... plenty of stuff everywhere (to be elaborated on)
poniedziałek, 1 sierpnia 2016
Top reading and exploring on my list. xgboost, ensembles, preprocessing.
1. xgboost - one of Kaggle winner packages, efficient code for home users.
http://www.r-bloggers.com/an-introduction-to-xgboost-r-package/
https://github.com/rachar1/DataAnalysis/blob/master/xgboost_Classification.R
Awesome XGBoost: https://github.com/dmlc/xgboost/blob/master/demo/README.md#features-walkthrough
https://github.com/dmlc/ xgboost/blob/master/R-package/ vignettes/xgboostPresentation. Rmd
http://wiselily.com/2015/07/12/xgboost-data-mining-example-1/
apply Xgboost in Kaggle with us https://m.youtube.com/watch?v= zwKFyMkvNXE
Tuning parameters:
https://rpubs.com/flyingdisc/ practical-machine-learning- xgboost
http://stackoverflow.com/ questions/33949735/tuning- xgboost-parameters-in-r
http://www.r-bloggers.com/r- setup-a-grid-search-for- xgboost/
https://www.kaggle.com/c/otto-group-product-classification-challenge/forums/t/14009/script-understanding-xgboost-model-on-otto-dataset
https://www.kaggle.com/tqchen/ otto-group-product- classification-challenge/ understanding-xgboost-model- on-otto-data/code
http://stats.stackexchange. com/questions/171043/how-to- tune-hyperparameters-of- xgboost-trees
https://www.kaggle.com/c/bnp- paribas-cardif-claims- management/forums/t/19922/how- to-tune-xgboost-using-r
https://www.kaggle.com/forums/ f/15/kaggle-forum/t/17120/how- to-tuning-xgboost-in-an- efficient-way
https://www.analyticsvidhya. com/blog/2016/01/xgboost- algorithm-easy-steps/
https://github.com/topepo/ caret/issues/336
Plot:
http://rpackages.ianhowson.com/cran/xgboost/man/xgb.plot.tree.html
(off topic, rearange corr) https://drsimonj.svbtle.com/rearrange-your-correlations-with-corrr
http://www.analyticsvidhya.com/blog/2016/01/xgboost-algorithm-easy-steps/
(also RF: http://www.analyticsvidhya.com/blog/2015/09/random-forest-algorithm-multiple-challenges/)
https://cran.r-project.org/web/packages/xgboost/vignettes/xgboostPresentation.html
https://www.kaggle.com/rajivranjansingh/liberty-mutual-group-property-inspection-prediction/xgboost-in-caret/run/44015/code
https://cran.r-project.org/web/packages/xgboost/vignettes/discoverYourData.html
https://cran.r-project.org/web/packages/xgboost/vignettes/xgboostPresentation.html
2. ensemble learning
http://www.vikparuchuri.com/blog/intro-to-ensemble-learning-in-r/
3. preprocessing
http://topepo.github.io/caret/preprocess.html
4. Another free course:
https://lagunita.stanford.edu/courses/HumanitiesScience/StatLearning/Winter2014/about
5. Deep learning in R resources: https://discuss. analyticsvidhya.com/t/deep- learning-in-r/10349
Do not read this, My first attempts:
library(xgboost)
mydata<-read.csv(file="clipboard", sep="\t", header=T)
mydata<-sapply(mydata, as.numeric)
#all columns as numeric
mydata<-as.data.frame(lapply(mydata,as.numeric)
#or
mydata<-as.data.frame(sapply(mydata[,1:3], as.numeric))
#label must be 0 or 1 in logistic regression
mydata$Gatunek<-mydata$Gatunek-1
model <- xgboost(data = as.matrix(mydata[,1:2]), label = mydata$Gatunek,
nrounds = 2, objective = "binary:logistic")
> install.packages("DiagrammeR")
xgb.plot.tree(model = model)
http://www.r-bloggers.com/an-introduction-to-xgboost-r-package/
https://github.com/rachar1/DataAnalysis/blob/master/xgboost_Classification.R
Awesome XGBoost: https://github.com/dmlc/xgboost/blob/master/demo/README.md#features-walkthrough
https://github.com/dmlc/
http://wiselily.com/2015/07/12/xgboost-data-mining-example-1/
apply Xgboost in Kaggle with us https://m.youtube.com/watch?v=
Tuning parameters:
https://rpubs.com/flyingdisc/
http://stackoverflow.com/
http://www.r-bloggers.com/r-
https://www.kaggle.com/c/otto-group-product-classification-challenge/forums/t/14009/script-understanding-xgboost-model-on-otto-dataset
https://www.kaggle.com/tqchen/
http://stats.stackexchange.
https://www.kaggle.com/c/bnp-
https://www.kaggle.com/forums/
https://www.analyticsvidhya.
https://github.com/topepo/
Plot:
http://rpackages.ianhowson.com/cran/xgboost/man/xgb.plot.tree.html
(off topic, rearange corr) https://drsimonj.svbtle.com/rearrange-your-correlations-with-corrr
http://www.analyticsvidhya.com/blog/2016/01/xgboost-algorithm-easy-steps/
(also RF: http://www.analyticsvidhya.com/blog/2015/09/random-forest-algorithm-multiple-challenges/)
https://cran.r-project.org/web/packages/xgboost/vignettes/xgboostPresentation.html
https://www.kaggle.com/rajivranjansingh/liberty-mutual-group-property-inspection-prediction/xgboost-in-caret/run/44015/code
https://cran.r-project.org/web/packages/xgboost/vignettes/discoverYourData.html
https://cran.r-project.org/web/packages/xgboost/vignettes/xgboostPresentation.html
2. ensemble learning
http://www.vikparuchuri.com/blog/intro-to-ensemble-learning-in-r/
3. preprocessing
http://topepo.github.io/caret/preprocess.html
4. Another free course:
https://lagunita.stanford.edu/courses/HumanitiesScience/StatLearning/Winter2014/about
5. Deep learning in R resources: https://discuss.
Do not read this, My first attempts:
library(xgboost)
mydata<-read.csv(file="clipboard", sep="\t", header=T)
mydata<-sapply(mydata, as.numeric)
#all columns as numeric
mydata<-as.data.frame(lapply(mydata,as.numeric)
#or
mydata<-as.data.frame(sapply(mydata[,1:3], as.numeric))
#label must be 0 or 1 in logistic regression
mydata$Gatunek<-mydata$Gatunek-1
model <- xgboost(data = as.matrix(mydata[,1:2]), label = mydata$Gatunek,
nrounds = 2, objective = "binary:logistic")
> install.packages("DiagrammeR")
xgb.plot.tree(model = model)
wtorek, 28 czerwca 2016
Free books: Exploring Data Science by John Mount and Nina Zumel
https://dzone.com/articles/practical-data-science-with-r
czwartek, 23 czerwca 2016
xgboost, caret, cleaning, time series - a reading list for next weeks.
Go through this to learn xgboost:
https://github.com/dmlc/xgboost/blob/master/R-package/vignettes/discoverYourData.Rmd
https://cran.r-project.org/ web/packages/xgboost/ vignettes/discoverYourData. html
https://github.com/dmlc/xgboost/blob/master/R-package/vignettes/xgboostPresentation.Rmd
https://github.com/dmlc/xgboost/tree/master/R-package/demo
https://rpubs.com/flyingdisc/practical-machine-learning-xgboost
http://courseprojects. souravsengupta.com/tag/ xgboost/
http://xgboost.readthedocs.io/ en/latest/R-package/ xgboostPresentation.html
https://www.kaggle.com/ jkapila/caterpillar-tube- pricing/0-24-with-xgboost-in- r/run/29813
http://www.analyticsvidhya. com/blog/2016/01/xgboost- algorithm-easy-steps/
Material from recent meetup in Warsaw:
https://github.com/mi2-warsaw/SER/blob/master/SER_XIX/xgboost.R
Cleaning and some interesting packages (editrules, deducorrect):
https://cran.r-project.org/doc/contrib/de_Jonge+van_der_Loo-Introduction_to_data_cleaning_with_R.pdf
http://www.r-bloggers.com/three-quick-and-simple-data-cleaning-helper-functions-december-2013/
http://www.meetup.com/amst-R-dam/events/57161682/ and especially:
http://cran.r-project.org/web/packages/editrules/index.html
http://cran.r-project.org/web/packages/deducorrect/index.html
Very good examples: https://github.com/data-cleaning
http://www.r-bloggers.com/deductive-imputation-with-the-deducorrect-package/
Time Series:
http://www.r-bloggers.com/time-series-analysis-and-mining-with-r/
Ensemble:
https://cran.r-project.org/web/packages/caretEnsemble/vignettes/caretEnsemble-intro.html
Caret: (very good stuff): https://github.com/pbiecek/DataMining/tree/master/MINI_2015
Other:
http://www.datasciencecentral. com/m/blogpost?id=6448529% 3ABlogPost%3A352821
Github new tools (also offline):
Write a book:
https://www.gitbook.com/
Collect and share snippets of code:
https://gist.github.com/
https://github.com/dmlc/xgboost/blob/master/R-package/vignettes/discoverYourData.Rmd
https://cran.r-project.org/
https://github.com/dmlc/xgboost/blob/master/R-package/vignettes/xgboostPresentation.Rmd
https://github.com/dmlc/xgboost/tree/master/R-package/demo
https://rpubs.com/flyingdisc/practical-machine-learning-xgboost
http://courseprojects.
http://xgboost.readthedocs.io/
https://www.kaggle.com/
http://www.analyticsvidhya.
Material from recent meetup in Warsaw:
https://github.com/mi2-warsaw/SER/blob/master/SER_XIX/xgboost.R
Cleaning and some interesting packages (editrules, deducorrect):
https://cran.r-project.org/doc/contrib/de_Jonge+van_der_Loo-Introduction_to_data_cleaning_with_R.pdf
http://www.r-bloggers.com/three-quick-and-simple-data-cleaning-helper-functions-december-2013/
http://www.meetup.com/amst-R-dam/events/57161682/ and especially:
http://cran.r-project.org/web/packages/editrules/index.html
http://cran.r-project.org/web/packages/deducorrect/index.html
Very good examples: https://github.com/data-cleaning
http://www.r-bloggers.com/deductive-imputation-with-the-deducorrect-package/
Time Series:
http://www.r-bloggers.com/time-series-analysis-and-mining-with-r/
Ensemble:
https://cran.r-project.org/web/packages/caretEnsemble/vignettes/caretEnsemble-intro.html
Caret: (very good stuff): https://github.com/pbiecek/DataMining/tree/master/MINI_2015
Other:
http://www.datasciencecentral.
Github new tools (also offline):
Write a book:
https://www.gitbook.com/
Collect and share snippets of code:
https://gist.github.com/
środa, 18 maja 2016
Excel interfacing R via RScript. (draft)
I played a bit, building a trifle application in Excel using R to explore weather data. My purpose was to make the code portable, with relative links - all files in one folder.
Here is my excel VBA code for exchanging data with R. It starts an R script and then reads back a txt file with numerical results and three plots as svg images.
And here is my R file which reads data from excel and then returns plots as images and reports in text format that VBA will paste in an Excel sheet. The commented install.packages need to be run once in R (maybe it needs to be corrected so that if the packages are not installed R should install them). Note the args... and then scriptPath... piece of the code which reads the path to the current location of the script. Not my solution - I will attach the source soon, sorry and thanks to the original author.
Here is my excel VBA code for exchanging data with R. It starts an R script and then reads back a txt file with numerical results and three plots as svg images.
Sub RunRscript2()
Dim shell As Object: Set shell = VBA.CreateObject("WScript.Shell")
Dim waitOnReturn As Boolean: waitOnReturn = True
Dim windowStyle As Integer: windowStyle = 1
Dim errorCode As Long
Dim sFile As String
'Cleanup
ThisWorkbook.Save
Worksheets("Sheet1").Activate
On Error Resume Next
Worksheets("Sheet1").QueryTables("output1").Delete
On Error GoTo 0
Worksheets("Sheet1").Pictures.Delete
'=========
sFile = ThisWorkbook.Path & "\test_xls_connectivity.R"
errorCode = shell.Run("C:\R\R-3.3.0\bin\x64\Rscript.exe " & sFile & "", windowStyle, waitOnReturn)
'errorCode = shell.Run("Rscript.exe " & sFile & "", windowStyle, waitOnReturn)
Worksheets("Sheet1").Range("$F$3").Select
sFile = ThisWorkbook.Path & "\plots.png"
ActiveSheet.Pictures.Insert(sFile).Select
With Selection.ShapeRange.Line
.Visible = msoTrue
.ForeColor.ObjectThemeColor = msoThemeColorText1
.ForeColor.TintAndShade = 0
.ForeColor.Brightness = 0
.Transparency = 0
End With
Worksheets("Sheet1").Range("$F$30").Select
sFile2 = ThisWorkbook.Path & "\plots2.png"
ActiveSheet.Pictures.Insert(sFile2).Select
With Selection.ShapeRange.Line
.Visible = msoTrue
.ForeColor.ObjectThemeColor = msoThemeColorText1
.ForeColor.TintAndShade = 0
.ForeColor.Brightness = 0
.Transparency = 0
End With
Worksheets("Sheet1").Range("$N$3").Select
sFile3 = ThisWorkbook.Path & "\plots3.png"
ActiveSheet.Pictures.Insert(sFile3).Select
With Selection.ShapeRange.Line
.Visible = msoTrue
.ForeColor.ObjectThemeColor = msoThemeColorText1
.ForeColor.TintAndShade = 0
.ForeColor.Brightness = 0
.Transparency = 0
End With
'''''''''''''''''''''''''
strFile = ThisWorkbook.Path & "\output.csv"
With ActiveSheet.QueryTables.Add(Connection:= _
"TEXT;" & strFile & "", Destination:=Range("$N$30"))
.Name = "output1"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.RefreshStyle = xlOverwriteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.TextFilePromptOnRefresh = False
.TextFilePlatform = 1252
.TextFileStartRow = 1
.TextFileParseType = xlDelimited
.TextFileTextQualifier = xlTextQualifierDoubleQuote
.TextFileConsecutiveDelimiter = False
.TextFileTabDelimiter = True
.TextFileSemicolonDelimiter = True
.TextFileCommaDelimiter = False
.TextFileSpaceDelimiter = False
.TextFileColumnDataTypes = Array(1, 1)
.TextFileTrailingMinusNumbers = True
.Refresh BackgroundQuery:=False
End With
End Sub
And here is my R file which reads data from excel and then returns plots as images and reports in text format that VBA will paste in an Excel sheet. The commented install.packages need to be run once in R (maybe it needs to be corrected so that if the packages are not installed R should install them). Note the args... and then scriptPath... piece of the code which reads the path to the current location of the script. Not my solution - I will attach the source soon, sorry and thanks to the original author.
# install.packages("xlsx")
# install.packages("rattle")
# install.packages("RColorBrewer")
# install.packages("rpart.plot")
# install.packages("scatterplot3d")
require(xlsx)
require(rpart)
# require(rattle)
require(rpart.plot)
require(ggplot2)
require(scatterplot3d)
rm(list=ls())
args <- commandArgs(trailingOnly = F)
scriptPath <- normalizePath(dirname(sub("^--file=", "", args[grep("^--file=", args)])))
setwd(scriptPath)
# setwd("C:\\Users\\jkotows2\\Desktop\\testR_xlsm")
#read from Sheet1
DaneBadawcze = read.xlsx("test_R.xlsm",1, encoding = "UTF-8", stringsAsFactors=T)
attach(DaneBadawcze)
fit <- rpart(Padało ~ Temperatura + Ciśnienie + Wilgotność,
,data=DaneBadawcze, method= "class")
#cluster<-cutree(hclust(dist(DaneBadawcze[,names(DaneBadawcze)!="Padało"]) ),k = 4)
cluster<-cutree(hclust(dist(DaneBadawcze) ),k = 4)
DaneBadawcze$Klaster<-cluster
#print a plot to a file
fn <- "plots.png"
if (file.exists(fn)) file.remove(fn)
png(fn)
#fancyRpartPlot(fit)
prp(fit,type = 2, fallen.leaves = T, extra = 3, compress = T)
dev.off()
#print another plot to a file
fn <- "plots2.png"
if (file.exists(fn)) file.remove(fn)
png(fn)
scatterplot3d(type="h", x=Temperatura, y=Ciśnienie, z=Wilgotność,color=c("green","blue") [DaneBadawcze$Padało] )
dev.off()
#print another plot to a file
fn <- "plots3.png"
if (file.exists(fn)) file.remove(fn)
png(fn)
scatterplot3d(type="h", x=Temperatura, y=Ciśnienie, z=Wilgotność,color=c("green","blue","red","brown","orange") [DaneBadawcze$Klaster] )
dev.off()
write.csv2(fit$variable.importance, file="output.csv", row.names = T)
#print a report to a file
sink(file="report.txt", type="output")
fit$variable.importance
unlink("report.txt")
sink()
To be continued.
piątek, 13 maja 2016
5graders learning R / R w 5tej klasie.
Read this: https://benaracreations2015.shinyapps.io/gr5project/
Implement the program in Polish schools!!!
(Found on Twitter @StatGarrett and @hadleywickham)
Implement the program in Polish schools!!!
(Found on Twitter @StatGarrett and @hadleywickham)
czwartek, 12 maja 2016
RStudio Addins
I am not so fast at memorising/coding... I need parallel code/(some)wysiwyg development. And my memory fails me, I need snippets management... Maybe addins will help soon. Newest RStudio offers addins functionality. Here are some:
https://github.com/daattali/addinslist#addinslist-table
https://github.com/daattali/addinslist#addinslist-table
sobota, 26 marca 2016
VBA: Simple and excellent example of using R from Excel
Acomed Statistik from Leipzig! Thanks for this very simple example of using R from Excel with VBA - here to calculate data for a kernel density plot. I recommend studying this simple example and will certainly try to adapt it to my needs. http://www.acomed-statistics.com/simple-interface-r-excel_engl.html
Another interesting tutorial to show how to drive R scripts from Excel. http://shashiasrblog.blogspot.com/2013/10/vba-front-end-for-r.html
The quickest setup for R available in Excel (UDF's) functions working on Excel ranges, and the easiest one: http://bert-toolkit.com/
eg. I wrote these in BERT config file (a file in R format) to get clustering functionality
Another interesting tutorial to show how to drive R scripts from Excel. http://shashiasrblog.blogspot.com/2013/10/vba-front-end-for-r.html
The quickest setup for R available in Excel (UDF's) functions working on Excel ranges, and the easiest one: http://bert-toolkit.com/
eg. I wrote these in BERT config file (a file in R format) to get clustering functionality
KMEANS_CLUST <- function( mat, ncenters ){ kmeans(mat,centers = ncenters) }
HTREE_CLUST<- function ( mat, cutheight ) {
myclust<-hclust(dist(mat));
myclust<-cutree(myclust,h=cutheight);
return(myclust)
}
---
środa, 23 marca 2016
A big idea for kid's math education.
Teach kids statistics? One hour that may decide about their whole life's success measured in their ability to understand, to research, to choose? And learn from them. Understand your limitations. Take an ultimate Uni exam. Absolutely mind-blowing experience of a project. "Statistics for 8-9-yr. olds or can you weigh a dog with a ruller?" http://smarterpoland.pl/index.php/2016/03/statystyka-dla-8-9-latkow-czyli-jak-zwazyc-psa-linijka/#comment-144319
What next? Here is one of the organizers Przemysław Biecek: https://www.youtube.com/watch?v=UEIcsVhFaSs, pity that it is addressed to Polish audience.You would be enchanted.
What next? R package for kids? Data exploration/machine learning for kids? It is already thought about:
https://github.com/pbiecek/BetaBit
Go SmarterPoland!
What next? Here is one of the organizers Przemysław Biecek: https://www.youtube.com/watch?v=UEIcsVhFaSs, pity that it is addressed to Polish audience.You would be enchanted.
What next? R package for kids? Data exploration/machine learning for kids? It is already thought about:
https://github.com/pbiecek/BetaBit
Go SmarterPoland!
wtorek, 8 marca 2016
R tutorials in Computerworld.
R in 5 lines series by Sharon Machlis: http://www.computerworld.com/tag/rin5linesorless/?nsdr=true#tk.ctw-infsb
czwartek, 11 lutego 2016
Coffee break: Lesson ideas: R and Titanic disaster - trees and forests.
Great ideas for a beginner like me to play with data mining.
Data and challenge proposed by kaggle.com:
https://www.kaggle.com/c/titanic/dat
Good tutorials and proposed solutins here:
http://trevorstephens.com/post/72916401642/titanic-getting-started-with-r and
https://github.com/trevorstephens/titanic
Also: https://www.kaggle.com/amoyakd/titanic/randomforest-method-v1-0
https://www.kaggle.com/c/titanic/forums
Some reading on R capabilities:
http://www.edureka.co/blog/implementation-of-decision-tree/
http://www.r-bloggers.com/a-brief-tour-of-the-trees-and-forests/
Other tutorials:
Iris data: http://rischanlab.github.io/RandomForest.html
(R Basics worth exploring: http://rischanlab.github.io/)
http://dni-institute.in/blogs/random-forest-using-r-step-by-step-tutorial/ (might be even better for a start
http://www.edureka.co/blog/implementation-of-decision-tree/
Biostars Tutorial: Machine Learning For Cancer Classification - Part 1 - Preparing The Data Sets
https://www.biostars.org/p/85124/
http://www.tutorialspoint.com/r/r_random_forest.htm
http://www.analyticsvidhya.com/blog/2015/09/random-forest-algorithm-multiple-challenges/
Other important topics to explore:
http://www.analyticsvidhya.com/blog/2015/12/faster-data-manipulation-7-packages/
Data and challenge proposed by kaggle.com:
https://www.kaggle.com/c/titanic/dat
Good tutorials and proposed solutins here:
http://trevorstephens.com/post/72916401642/titanic-getting-started-with-r and
https://github.com/trevorstephens/titanic
Also: https://www.kaggle.com/amoyakd/titanic/randomforest-method-v1-0
https://www.kaggle.com/c/titanic/forums
Some reading on R capabilities:
http://www.edureka.co/blog/implementation-of-decision-tree/
http://www.r-bloggers.com/a-brief-tour-of-the-trees-and-forests/
Other tutorials:
Iris data: http://rischanlab.github.io/RandomForest.html
(R Basics worth exploring: http://rischanlab.github.io/)
http://dni-institute.in/blogs/random-forest-using-r-step-by-step-tutorial/ (might be even better for a start
http://www.edureka.co/blog/implementation-of-decision-tree/
Biostars Tutorial: Machine Learning For Cancer Classification - Part 1 - Preparing The Data Sets
https://www.biostars.org/p/85124/
http://www.tutorialspoint.com/r/r_random_forest.htm
http://www.analyticsvidhya.com/blog/2015/09/random-forest-algorithm-multiple-challenges/
Other important topics to explore:
http://www.analyticsvidhya.com/blog/2015/12/faster-data-manipulation-7-packages/
niedziela, 7 lutego 2016
Small plots on a plot: R
Pie charts instead of points or bubbles in x-y/scatter-plot/bubble plot:
Uncle google found this for me:
http://blog.revolutionanalytics.com/2012/09/visualize-complex-data-with-subplots.html
http://www.r-bloggers.com/embeding-a-subplot-in-ggplot-via-subview/
http://vita.had.co.nz/papers/embedded-plots.pdf
ggsubplot
http://stackoverflow.com/questions/26577669/tiny-pie-charts-to-represent-each-point-in-an-scatterplot-using-ggplot2
http://stackoverflow.com/questions/7714677/r-scatterplot-with-too-many-points/16122003#16122003
pieGlyph
http://stackoverflow.com/questions/20465070/barplots-on-a-map/20468836#20468836
Maybe it will be possible in ggvis??
http://ggvis.rstudio.com/ggvis-basics.html
===========
Here a solution without ggplot (p.11):
https://www.r-project.org/doc/Rnews/Rnews_2003-2.pdf
====
plotrix and floating.pie?
http://ask.programmershare.com/2269_10385068/
====
http://gis.stackexchange.com/questions/4568/effectively-displaying-demographic-data-on-a-printed-map
====
ggtree
http://stackoverflow.com/questions/10368180/plotting-pie-graphs-on-map-in-ggplot
mapplots
http://www.molecularecologist.com/2012/09/making-maps-with-r/
Rozwiazanie w Excelu:
http://www.andypope.info/charts/piedatamarkers.htm
http://blog.revolutionanalytics.com/2012/09/visualize-complex-data-with-subplots.html
http://www.r-bloggers.com/embeding-a-subplot-in-ggplot-via-subview/
http://vita.had.co.nz/papers/embedded-plots.pdf
ggsubplot
http://stackoverflow.com/questions/26577669/tiny-pie-charts-to-represent-each-point-in-an-scatterplot-using-ggplot2
http://stackoverflow.com/questions/7714677/r-scatterplot-with-too-many-points/16122003#16122003
pieGlyph
http://stackoverflow.com/questions/20465070/barplots-on-a-map/20468836#20468836
Maybe it will be possible in ggvis??
http://ggvis.rstudio.com/ggvis-basics.html
===========
Here a solution without ggplot (p.11):
https://www.r-project.org/doc/Rnews/Rnews_2003-2.pdf
====
plotrix and floating.pie?
http://ask.programmershare.com/2269_10385068/
====
http://gis.stackexchange.com/questions/4568/effectively-displaying-demographic-data-on-a-printed-map
====
ggtree
http://stackoverflow.com/questions/10368180/plotting-pie-graphs-on-map-in-ggplot
mapplots
http://www.molecularecologist.com/2012/09/making-maps-with-r/
Rozwiazanie w Excelu:
http://www.andypope.info/charts/piedatamarkers.htm
poniedziałek, 1 lutego 2016
Coffee break: imputation
From CranR:
Simple mean imputation: http://www.r-bloggers.com/example-2014-5-simple-mean-imputation/
knnImputation {DMwR} Function that fills in all NA values using the k Nearest Neighbours of each case with NA values.
centralImputation {DMwR} This function fills in any NA value in all columns of a data frame with the statistic of centrality (given by the function centralvalue()) of the respective column.
Other: kNNImpute {imputation} Imputation using k-nearest neighbors. For each record, identify missinng features. http://www.bioconductor.org/packages/release/bioc/html/impute.html
impute {Hmisc} http://www.inside-r.org/packages/cran/hmisc/docs/impute http://www.jstatsoft.org/article/view/v023i10 http://docs.zeligproject.org/en/latest/using-Zelig-with-Amelia.html
Simple mean imputation: http://www.r-bloggers.com/example-2014-5-simple-mean-imputation/
knnImputation {DMwR} Function that fills in all NA values using the k Nearest Neighbours of each case with NA values.
centralImputation {DMwR} This function fills in any NA value in all columns of a data frame with the statistic of centrality (given by the function centralvalue()) of the respective column.
Other: kNNImpute {imputation} Imputation using k-nearest neighbors. For each record, identify missinng features. http://www.bioconductor.org/packages/release/bioc/html/impute.html
impute {Hmisc} http://www.inside-r.org/packages/cran/hmisc/docs/impute http://www.jstatsoft.org/article/view/v023i10 http://docs.zeligproject.org/en/latest/using-Zelig-with-Amelia.html
piątek, 29 stycznia 2016
What u play with on smartphone? I play R stats :-)
I followed this tut, thanks! http://www.r-bloggers.com/install-r-in-android-via-gnuroot-no-root-required/
1. Installed GnuRoot app and GnuRoot Wheezy (no real rooting required, it creates its own world!)
2. Refreshed software: apt-get update & apt-get upgrade.
3. Installed R: apt-get install R-base-core. Lots of space is needed, so delete all trash apps from your android phone!
===========
I typed R, created a dataframe to play with, run kmeans on it, with no issues. There is no graphics yet, you will not see plots (probably you can generate them as files), unless you install another heavy MB of linux GUI like lxde. But for my bus trips I use txtplot - it produces density, boxplot, scatter, acf in ASCII.
To see kmeans clusters: txtplot(x,y,pch=cluster_no)
R used phone's internet and downloaded it with no issues - it means you can also download dataframes from a repository online.
A keyboard with arrows is recommended, like Hacker's Keyboard, so that it should be easy to go through previous commands, modify them and correct typo's inside the string..
GnuRoot autor Corbin Champion also offers a solution to run Octave on your phone. I highly recommend him. https://play.google.com/store/apps/details?id=champion.gnuroot
More: Google: install r in android:
http://www.r-ohjelmointi.org/?p=1434
http://stackoverflow.com/questions/13047994/are-there-any-android-version-of-r-without-rooting-the-device
1. Installed GnuRoot app and GnuRoot Wheezy (no real rooting required, it creates its own world!)
2. Refreshed software: apt-get update & apt-get upgrade.
3. Installed R: apt-get install R-base-core. Lots of space is needed, so delete all trash apps from your android phone!
===========
I typed R, created a dataframe to play with, run kmeans on it, with no issues. There is no graphics yet, you will not see plots (probably you can generate them as files), unless you install another heavy MB of linux GUI like lxde. But for my bus trips I use txtplot - it produces density, boxplot, scatter, acf in ASCII.
To see kmeans clusters: txtplot(x,y,pch=cluster_no)
R used phone's internet and downloaded it with no issues - it means you can also download dataframes from a repository online.
A keyboard with arrows is recommended, like Hacker's Keyboard, so that it should be easy to go through previous commands, modify them and correct typo's inside the string..
GnuRoot autor Corbin Champion also offers a solution to run Octave on your phone. I highly recommend him. https://play.google.com/store/apps/details?id=champion.gnuroot
More: Google: install r in android:
http://www.r-ohjelmointi.org/?p=1434
http://stackoverflow.com/questions/13047994/are-there-any-android-version-of-r-without-rooting-the-device
czwartek, 14 stycznia 2016
wtorek, 5 stycznia 2016
R: xts and zoo package for missing data; two plots
Dealing with incomplete data:
Useful and basic entry into the topic:
http://nicercode.github.io/intro/missing-data.html and
http://thomasleeper.com/Rcourse/Tutorials/NAhandling.html
http://www.ats.ucla.edu/stat/r/faq/missing.htm
==Fill NA==
A very interesting reading on the use of zoo/xts package here (page 1 and the following): http://publish.illinois.edu/spencer-guerrero/2014/12/11/2-dealing-with-missing-data-in-r-omit-approx-or-spline-part-1/
My dirty notebook:
> is.na (
> mean(Data&Variable1, na.rm=TRUE)
Remove incomplete
> na.omit(Data$Variable1)
> complete.cases(Data)
> complete.data <- complete.cases(Data)
Fill continuous/time series data
> require (xts)
> prices <-na.locf(prices)
> prices <-na.fill(prices, fill="extend") are there other fills?
check out na.approx, na.spline!!!
na.contiguous (to find longest contiguous na's?)
For plotting
na.fill etc functions turn data frame into atomic vectors...
It means that if we plot, we cannot refer to column names but to column numbers.
Plot several plots:
> par(mfrow=(c(2,1))
> plot(prices[,1], type='l')
> plot(prices[,2], type='l')
====
For correlation
cor ()... use the argument use='pairwise', then only the pairs with na will be ommited and not the whole observations.
====
Serious approach: imputation:
http://www.stat.columbia.edu/~gelman/arm/missing.pdf
e.g.: http://stackoverflow.com/questions/11990463/match-fitted-values-from-lm-with-a-data-frame-in-case-of-na-values
https://stat.ethz.ch/pipermail/r-help/2002-January/017756.html
Good article for a study: http://www.r-bloggers.com/imputing-missing-data-with-r-mice-package/
http://research.stowers-institute.org/efg/R/Statistics/loess.htm
http://www.biostat.jhsph.edu/~iruczins/teaching/jf/ch14.pdf
http://www.ats.ucla.edu/stat/r/faq/R_pmm_mi.htm
http://www.stefvanbuuren.nl/publications/MICE%20in%20R%20-%20Draft.pdf
http://stackoverflow.com/questions/2613420/handling-missing-incomplete-data-in-r-is-there-function-to-mask-but-not-remove
Useful and basic entry into the topic:
http://nicercode.github.io/intro/missing-data.html and
http://thomasleeper.com/Rcourse/Tutorials/NAhandling.html
http://www.ats.ucla.edu/stat/r/faq/missing.htm
==Fill NA==
A very interesting reading on the use of zoo/xts package here (page 1 and the following): http://publish.illinois.edu/spencer-guerrero/2014/12/11/2-dealing-with-missing-data-in-r-omit-approx-or-spline-part-1/
My dirty notebook:
> is.na (
> mean(Data&Variable1, na.rm=TRUE)
Remove incomplete
> na.omit(Data$Variable1)
> complete.cases(Data)
> complete.data <- complete.cases(Data)
Fill continuous/time series data
> require (xts)
> prices <-na.locf(prices)
> prices <-na.fill(prices, fill="extend") are there other fills?
check out na.approx, na.spline!!!
na.contiguous (to find longest contiguous na's?)
For plotting
na.fill etc functions turn data frame into atomic vectors...
It means that if we plot, we cannot refer to column names but to column numbers.
Plot several plots:
> par(mfrow=(c(2,1))
> plot(prices[,1], type='l')
> plot(prices[,2], type='l')
====
For correlation
cor ()... use the argument use='pairwise', then only the pairs with na will be ommited and not the whole observations.
====
Serious approach: imputation:
http://www.stat.columbia.edu/~gelman/arm/missing.pdf
e.g.: http://stackoverflow.com/questions/11990463/match-fitted-values-from-lm-with-a-data-frame-in-case-of-na-values
https://stat.ethz.ch/pipermail/r-help/2002-January/017756.html
Good article for a study: http://www.r-bloggers.com/imputing-missing-data-with-r-mice-package/
http://research.stowers-institute.org/efg/R/Statistics/loess.htm
http://www.biostat.jhsph.edu/~iruczins/teaching/jf/ch14.pdf
http://www.ats.ucla.edu/stat/r/faq/R_pmm_mi.htm
http://www.stefvanbuuren.nl/publications/MICE%20in%20R%20-%20Draft.pdf
http://stackoverflow.com/questions/2613420/handling-missing-incomplete-data-in-r-is-there-function-to-mask-but-not-remove
poniedziałek, 4 stycznia 2016
Maps of Poland in R
To start with:
http://www.gauss.pl/analityk/drupal/node/43
http://www.kep.uni.lodz.pl/em/materialy/BANKI/BANKI_DANYCH_cw_10.doc
Most comprehensive:
http://quantup.pl/2015/03/13/analiza-danych/wizualizacja-danych-na-mapach-w-r/
http://www.goldenline.pl/grupy/Komputery_Internet/r/mapa-polski,1523596/
http://www.gis-support.pl/baza-wiedzy/dane/dane-do-pobrania/
http://www.wais.kamil.rzeszow.pl/pakiet-r-openpoland-tutorial/
In Excel
http://www.apocotenexcel.pl/mapa2.htm
http://www.gauss.pl/analityk/drupal/node/43
http://www.kep.uni.lodz.pl/em/materialy/BANKI/BANKI_DANYCH_cw_10.doc
Most comprehensive:
http://quantup.pl/2015/03/13/analiza-danych/wizualizacja-danych-na-mapach-w-r/
http://www.goldenline.pl/grupy/Komputery_Internet/r/mapa-polski,1523596/
http://www.gis-support.pl/baza-wiedzy/dane/dane-do-pobrania/
http://www.wais.kamil.rzeszow.pl/pakiet-r-openpoland-tutorial/
In Excel
http://www.apocotenexcel.pl/mapa2.htm
środa, 30 grudnia 2015
Day 2 with R. Ideas for diving in deep waters
Start with example here: https://stat.ethz.ch/R-manual/R-devel/library/class/html/knn.html
Then go through: http://blog.datacamp.com/machine-learning-in-r/
========================
Additional reading:
https://en.wikibooks.org/wiki/Data_Mining_Algorithms_In_R/Classification/kNN
https://www3.nd.edu/~steve/computing_with_data/17_Refining_kNN/refining_knn.html
http://blog.webagesolutions.com/archives/1164
http://www.analyticsvidhya.com/blog/2015/08/learning-concept-knn-algorithms-programming/
K means vs K nearest neighbors are two different things.
https://www.quora.com/Is-the-k-Means-algorithm-related-to-the-k-Nearest-Neighbors-algorithm
https://www.quora.com/How-is-the-k-nearest-neighbor-algorithm-different-from-k-means-clustering
What next:
http://rayli.net/blog/data/top-10-data-mining-algorithms-in-plain-r/
Additional topic:
R and SQL/MS SQL
http://www.burns-stat.com/translating-r-sql-basics/
http://www.r-bloggers.com/make-r-speak-sql-with-sqldf/ - use SQL in R
https://www.simple-talk.com/sql/reporting-services/making-data-analytics-simpler-sql-server-and-r/ make R talk to MS SQL
Ideas for day 3: http://courses.had.co.nz/10-tokyo/
http://www.analyticsvidhya.com/learning-paths-data-science-business-analytics-business-intelligence-big-data/learning-path-r-data-science/
Subskrybuj:
Posty (Atom)