1 – Lessons begin

Welcome to Coding-School.com old school!

Some of the tips, techniques and code freely imparted here, I’ve searched the web previously and come back empty handed. Then set about hand crafting my own solutions – which you get completely for free!.

If you have found my website useful, please consider buying me a coffee below 😉

Cheers and enjoy,

Mark

R – native and ggplot boxplots

date <- seq.Date(as.Date("2013-06-01"), as.Date("2014-05-31"), "days")
set.seed(100)
x <- as.integer(abs(rnorm(365))*1000)
df <- data.frame(date, x)
boxplot(df$x ~ months(df$date), outline = FALSE,las=2)
library(ggplot2)

ggplot(df) +
  geom_boxplot(aes(reorder(format(date,'%B'),date),fill=format(date,'%Y'),x)) +
        xlab('Month') + guides(fill=guide_legend(title="Year")) +
        theme(axis.text.x = element_text(angle = 45)) 

R – ggplot example

$ head dl.csv 
time,code,count
6:59,200,31
7:00,200,1841
7:00,502,3644
7:01,200,369

> x<-read.csv("dl.csv")
> library(dplyr)
> library(tidyverse)

> ggplot(x,aes(time,count,color=code))+geom_point()+scale_x_discrete(breaks = levels(x$time)[c(T, rep(F, 5))])+theme(axis.text.x = element_text(angle = 90, vjust = 0.5, hjust=1))

R – graph by month

> options(scipen=5)
> jan20<-x[grepl("2023-01",x$date) & x$enabled > 0 & x$ignore==0,]
> barplot(tapply(jan20$users,jan20$date,sum)/1000,las=2,main="Jan 2023 - by month",col=rainbow(10),cex.names=0.8)

R – Read in CSV

> setwd("/Users/mark/Documents/Stats")
> x<-read.csv("sites.csv",sep="\t",stringsAsFactors = F)
> summary(x$url)
Length     Class      Mode 
   983002 character character
 
> summary(x$score)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
 -3.000  -2.000  -1.000   1.977  -1.000  25.000       1 

R – Package List

How to list all packages that have been imported into current session

> (.packages())

Example output

> (.packages())
[1] "stats"     "graphics"  "grDevices" "utils"     "datasets"  "methods"  
[7] "base" 

To see if a package is installed:

> x<-grep("tidyverse",installed.packages())
> installed.packages()[x]

Install a number of packages as below:

> install.packages(c("nycflights13", "gapminder", "Lahman"))

Instances – AWS CLI

  1. aws ec2 describe-instances | jq -cr ‘.Reservations[].Instances[0]|[.InstanceId,.InstanceType,.KeyName,.State,.Tags]’ # list instances
  2. aws ec2 describe-instance-status –instance-id <INST-ID> | jq ‘ .InstanceStatuses | .[].InstanceState.Name’ # show instance
  3. aws ec2 start-instances –instance-id <INST-ID> # start instance

LoadBalancers – AWS CLI

  1. aws elbv2 describe-load-balancers| jq -c ‘.LoadBalancers[]|[.LoadBalancerName,.LoadBalancerArn]’ # list arns
  2. aws elbv2 describe-listeners –load-balancer-arn <ARN>  | jq -c ‘.|.Listeners[]|[.Protocol,.Port,.ListenerArn] # list listeners
  3.  aws elbv2 describe-rules –listener-arn <ARN> | jq -c ‘.Rules[]|.Conditions,.Actions’ # list rules

MongoDB Intro

  1. Connect to mongodb: $ mongo
  2. show dbs;
  3. db.<collection>.stats(1024000000) # gb – ref
  4. db.<collection>.find({},{<field>:<value>}).sort({<field>:1}).limit(5); # find top 5
  5. db.<collection>.find({},{<field>:1}).sort({<field>:-1}).limit(5); # find bottom 5
  6. db.<collection>.find({<field>:{ $lt: <val>}},{<field>:1}).count(); # count matches
  7. db.runCommand({compact:'<collection>’}) # compact collection

Cassandra – tablestats

How to produce a digest of tables using nodetool

$ nodetool tablestats | awk ‘ /Keyspace/ || /Latency/ {print $0} /Table:/ {gsub(/^[ ][ ]/,””);table=”–> “$0} /Space used/ && $NF > 0 {gsub(/^[ ][ ]/,””);space=space “|” $0} /used by snapshots/ {printf(“%-30s\t%-30s\n”,table,space);table=””;space=””} ‘

Keyspace : system_traces
Read Latency: NaN ms
Write Latency: NaN ms
–> Table: events
–> Table: sessions
Keyspace : system
Read Latency: 4.279434782608696 ms
Write Latency: 0.3454382659499075 ms
–> Table: IndexInfo
–> Table: available_ranges
–> Table: batches
–> Table: batchlog
–> Table: built_views
–> Table: compaction_history |Space used (live): 13424|Space used (total): 13424
–> Table: hints
–> Table: local |Space used (live): 15977|Space used (total): 15977
–> Table: paxos
–> Table: peer_events
–> Table: peers |Space used (live): 14912|Space used (total): 14912
–> Table: prepared_statements
–> Table: range_xfers
–> Table: size_estimates |Space used (live): 94270|Space used (total): 94270
–> Table: sstable_activity |Space used (live): 11098|Space used (total): 11098
–> Table: transferred_ranges
–> Table: views_builds_in_progress
Keyspace : system_distributed
Read Latency: NaN ms
Write Latency: NaN ms
–> Table: parent_repair_history
–> Table: repair_history
–> Table: view_build_status
Keyspace : system_schema
Read Latency: 1.7939485294117647 ms
Write Latency: 2.188590909090909 ms
–> Table: aggregates
–> Table: columns |Space used (live): 18096|Space used (total): 18096
–> Table: dropped_columns
–> Table: functions
–> Table: indexes
–> Table: keyspaces |Space used (live): 10828|Space used (total): 10828
–> Table: tables |Space used (live): 16857|Space used (total): 16857
–> Table: triggers
–> Table: types
–> Table: views
Keyspace : system_auth
Read Latency: 0.241 ms
Write Latency: 0.123 ms
–> Table: resource_role_permissons_index
–> Table: role_members
–> Table: role_permissions
–> Table: roles |Space used (live): 5134|Space used (total): 5134

Python Lesson #2

One of the first things you are probably going to want to do is use modules.

For example to print out pi, we use the math module as below

import math
math.pi

3.141592653589793

When we import modules they generally come with same namespace, i.e. calling pi function with prefix namespace of math. For performance, etc we can also just import the specific functions we want to use. This puts the function in the global name space. For example:

from math import pi
pi

It is also possible to use a alias – such as

import math as m
m.pi

You can also create your own modules simply by storing functions in filenames with .py extension. So for example we create a file called mymodule.py with following content

# mymodule.py
def double(number):
—>return 2 * number

Then we invoke with following .. if you get an error try import sys;print sys.path for folders searched for modules. Also you can use PYTHONPATH system variable to include other paths

import mymodle
double(65)
>> 130