ShellScript 的一些实用使用方法

本文章将持续更新，记录一些在开发过程中实用的 Shell 脚本。

Linux 环境初始化

每次新建一个虚拟机或者登录到一个陌生的物理机时，都需要从头开始构建开发环境。这里提供一个 Shell 脚本，可以自动下载 JDK、Python、Oh-My-Zsh 等工具。

#!/bin/bash

# install tools
sudo apt-get update
sudo apt-get install -y python3-dev python3-pip python3-setuptools
pip3 install thefuck --user
sudo apt-get install -y zsh
sudo apt-get install -y openjdk-8-jdk 
# install vscode
sudo apt install -y software-properties-common apt-transport-https wget
wget -q https://packages.microsoft.com/keys/microsoft.asc -O- | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main"
sudo apt install code

# ssh
if [ ! -e $HOME/.ssh ];
then
mkdir -p $HOME/.ssh
fi

if [ ! -e $HOME/.ssh/authorized_keys ];
then
touch $HOME/.ssh/authorized_keys
fi

echo "your public key" >> ~/.ssh/authorized_keys

# setup zsh
echo 'export PATH=$PATH:$HOME/.local/bin' >> $HOME/.zshrc
chsh -s /bin/zsh
source ~/.zshrc

# install oh-my-zsh
while [ ! -e $HOME/.oh-my-zsh ];
do
    sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"
done

# installing oh-my-zsh-plugins
while [ ! -e $HOME/.oh-my-zsh ];
do
    sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"
done

while [ ! -e  $HOME/.oh-my-zsh/custom/plugins/zsh-autosuggestions ];
do
    git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions
done

while [ ! -e $HOME/.oh-my-zsh/custom/plugins/zsh-syntax-highlighting ];
do
    git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting
done

while [ ! -e $HOME/.oh-my-zsh/custom/plugins/zsh-z ];
do
    git clone https://github.com/agkozak/zsh-z ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-z
done

sed -i "s/plugins=.*/plugins=(git vscode zsh-autosuggestions zsh-syntax-highlighting z)/" $HOME/.zshrc

source ~/.zshrc

文件大小统计

这个脚本用于统计在 -d 目录下符合 -p 模式文件名的文件的数目以及总大小，通常用于统计用户线上 TsFile 的总大小、总数目以及平均大小。

#!/bin/bash

total_files=0
total_size=0
pattern=*.tsfile

while getopts "d:p:" opt; do
  case $opt in
    d)
      folders=$OPTARG
      ;;
    p)
      pattern=$OPTARG
      ;;
    *)
      echo "Usage: $0 -d <comma-separated-directories>"
      exit 1
      ;;
  esac
done

# 使用 IFS（内部字段分隔符）分割字符串为数组
IFS=',' read -ra folders_array <<< "$folders"

for folder in "${folders_array[@]}"; do
  count=`find "$folder" -type f -name "$pattern" -printf "%s\n"|wc -l`
  res=`find "$folder" -type f -name "$pattern" -printf "%s\n" | awk '{ sum += $1 } END { print sum }'`
  total_files=$((total_files + count))
  total_size=$((total_size + res))
done

if [ "$total_files" -ne 0 ]; then
  average_size=$((total_size / total_files))
  echo "Overall Total files: $total_files, $(echo "scale=2; $total_size / 1048576" | bc) MB"
  echo "Average file size: $(echo "scale=2; $average_size / 1048576" | bc) MB"
else
  echo "No files found."
fi

用法：

$ bash ./test.sh -d /data1/lxx,/data2/lxx,/data3/lxx -p "*-*-2-*.tsfile"
Overall Total files: 695, 1329241.62 MB
Average file size: 1912.57 MB