Proxy in terminal

ssh config

linux下通过按照如下修改.ssh/config设置账号密码，并 ssh -vT git@github.com，成功后输出Hi Kirrito-k423! You've successfully authenticated, but GitHub does not provide shell access.。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# .ssh/config
Host github.com
  User 943648187@qq.com
  Hostname ssh.github.com
  PreferredAuthentications publickey
  ProxyCommand nc -X 5 -x 127.0.0.1:7890 %h %p #如果通过代理需要这句话
  IdentityFile ~/.ssh/id_rsa
  Port 443

Host *
  # Win报错取消下面三行 getpeername failed: Not a socket getsockname failed: Not a socket
  ControlMaster auto
  ControlPath /tmp/sshcontrol-%C
  ControlPersist 1d
  ServerAliveInterval 30

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

Host github.com
  User git
  Port 22
  Hostname github.com
  # 注意修改路径为你的路径
  IdentityFile "C:\Users\Administrator\.ssh\id_rsa"
  TCPKeepAlive yes
  # 这里的 -a none 是 NO-AUTH 模式，参见 https://bitbucket.org/gotoh/connect/wiki/Home 中的 More detail 一节
  ProxyCommand E:\\commonSoftware\\Git\\mingw64\\bin\\connect.exe -S 127.0.0.1:7890 -a none %h %p

Host ssh.github.com
  User git
  Port 443
  Hostname ssh.github.com
  # 注意修改路径为你的路径
  IdentityFile "C:\Users\Administrator\.ssh\id_rsa"
  TCPKeepAlive yes

debug ssh clone/push

1
2
3
4

git config --global http.proxy localhost:7890 # PowerShell proxy
git config --global http.proxy "http://127.0.0.1:7890"
git config --global https.proxy "http://127.0.0.1:7890"
GIT_CURL_VERBOSE=1 GIT_TRACE=1 git clone git@github.com:Kirrito-k423/autoUpdateIpconfigPushGithub.git

1

GIT_CURL_VERBOSE=1 GIT_TRACE=1 git clone https://github.com/llvm/llvm-project.git

1
2
3

$env:GIT_CURL_VERBOSE = 1
$env:GIT_TRACE = 1
git push

1
2

$env:GIT_SSH_COMMAND = 'ssh -F /path/to/your/ssh_config'
git push

http代理

There are tons of identical solutions over the internet for defining proxy tunnel for git’s downloads like this one, which all is by setting git’s https.proxy & http.proxy config. but those answers are not working when you try to clone/push/pull etc. over the ssh protocol!

For example, by setting git config --global https.proxy socks5://127.0.0.1:9999 when you try to clone git clone git@github.org:user/repo.git it does not go through the defined sock5 tunnel!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

# Method 1. git http + proxy http
git config --global http.proxy "http://127.0.0.1:1080"
git config --global https.proxy "http://127.0.0.1:1080"

# Method 2. git http + proxy shocks
git config --global http.proxy "socks5://127.0.0.1:1080"
git config --global https.proxy "socks5://127.0.0.1:1080"

# to unset
git config --global --unset http.proxy
git config --global --unset https.proxy

# Method 3. git ssh + proxy http
vim ~/.ssh/config
Host github.com
HostName github.com
User git
ProxyCommand socat - PROXY:127.0.0.1:%h:%p,proxyport=1087

# Method 4. git ssh + proxy socks
vim ~/.ssh/config
Host github.com
HostName github.com
User git
ProxyCommand nc -v -x 127.0.0.1:1080 %h %p

%h %p 是host和post的意思

或者

After some visiting so many pages, I finally find the solution to my question:

1
2
3
4
5
6
7
8
9
10
11
12

# [step 1] create a ssh-proxy
  ssh -D 9999 -qCN user@server.net

# [step 2] make git connect through the ssh-proxy
  # [current script only]
  export GIT_SSH_COMMAND='ssh -o ProxyCommand="connect -S 127.0.0.1:9999 %h %p"'
  # OR [git global setting] 
  git config --global core.sshCommand 'ssh -o ProxyCommand="connect -S 127.0.0.1:9999 %h %p"'
  # OR [one-time only use]
  git clone -c=core.sshCommand 'ssh -o ProxyCommand="connect -S 127.0.0.1:9999 %h %p"' git@github.com:user/repo.git
  # OR [current repository use only]
  git config core.sshCommand 'ssh -o ProxyCommand="connect -S 127.0.0.1:9999 %h %p"'

To install connect on Ubuntu:

1

sudo apt install connect-proxy

ssh代理

1
2
3
4
5

ssh -vT -o "ProxyCommand connect -S 127.0.0.1:7890 %h %p" git@github.com
ssh -vT -o "ProxyCommand nc -X 5 -x 127.0.0.1:7890 %h %p" git@github.com
# 使用HTTP 代理
ssh -o ProxyCommand='corkscrew proxy.net 8888 %h %p' user@server.net
ssh -o ProxyCommand='proxytunnel -p proxy.net:8888 -P username -d %h:%p' user@server.net

Wireguard 代理

post request forward is an all-in-one solution.

1
2
3
4
5
6
7
8
9
10
11

interface: warp
  public key: fcDZCrGbcpz3sKFqhBw7PtdInygUOtEJfPAs08Wwplc=
  private key: (hidden)
  listening port: 51825

peer: bmXOC+F1FxEMF9dyiK2H5/1SUtzH0JuVo51h2wPfgyo=
  endpoint: [2606:4700:d0::a29f:c001]:1701
  allowed ips: 172.16.0.0/24, 0.0.0.0/0, ::/0
  latest handshake: 89 days, 23 hours, 15 minutes, 28 seconds ago
  transfer: 3.51 GiB received, 1.71 GiB sent
  persistent keepalive: every 25 seconds

latest handshake: 89 days ago demonstrate wg is done for a long time. At the same time mtr github.com shows no output prove the bad situation.

STEP1: first try is to bring the wg-proxy up again

1
2
3
4

python register.py #自动生成warp-op.conf,warp.conf和warp-helper
mv warp-helper /etc/default
vim /etc/config/network #填写warp-op.conf内容，修改只用替换option private_key 和 ipv6 的 list addresses 即可
ifup warp #启动warp, 代替wg-quick up warp.conf

and test brainiac machine is back online

常见情形

大文件提交

Sometimes，it‘s the big log fault.

1
2
3
4
5
6
7
8
9
10
11

# find file
find . -type f -name "zsim.log.0" -size +10M
# find the most repeated lines
head -n 10000 your_file.txt | sort | uniq -c | sort -nr | head
# delete partten line in files
sed -i '/\[S 0\] WARN: \[6\] ContextChange, reason SIGRETURN, inSyscall 1/d' /staff/shaojiemike/github/PIA_huawei/log/zsim/chai-n/hsti/1000/cpu_tlb/zsim.log.0

# conbine two command
find . -type f -name "zsim.log.0" -size +10M -print0 | xargs -0 sed -i '/字符串模式/d'
# or just save the tail (sth wrong needed test)
find . -type f -name "zsim.log.0" -size +1M -exec bash -c 'tail -n 2000 "$1" > "$1"_back ' _ {} \;

请求被拦截

1
2
3
4

t00906153@A2305023964 MINGW64 ~/github
$ git clone https://github.com/jeremy-rifkin/cpptrace.git
Cloning into 'cpptrace'...
fatal: unable to access 'https://github.com/jeremy-rifkin/cpptrace.git/': SSL certificate problem: self-signed certificate in certificate chain

常见问题

1
2
3
4
5
6
7
8
9
10

eno0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 202.38.73.217  netmask 255.255.255.0  broadcast 202.38.73.255
        inet6 fe80::ae1f:6bff:fe8a:e4ba  prefixlen 64  scopeid 0x20<link>
        inet6 2001:da8:d800:811:ae1f:6bff:fe8a:e4ba  prefixlen 64  scopeid 0x0<global>
        inet6 2001:da8:d800:730:ae1f:6bff:fe8a:e4ba  prefixlen 64  scopeid 0x0<global>
        ether ac:1f:6b:8a:e4:ba  txqueuelen 1000  (以太网)
        RX packets 12345942  bytes 2946978044 (2.9 GB)
        RX errors 0  dropped 1438318  overruns 0  frame 0
        TX packets 4582067  bytes 675384424 (675.3 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

1

ifconfig eno0 mtu 1200

1
2
3
4

[root@localhost ~]# vi /etc/sysconfig/network-scripts/ifcfg-eno0
MTU=1200            #MTU设置

[root@localhost ~]# systemctl restart network

参考文献

简介

• Cloudflare 作为全球最大的网络服务提供商，全球大约有30%的网络服务由它提供。
• WARP是Cloudflare提供的免费的，没有带宽上限或限制的VPN服务，可以让你在任何地方访问互联网，而不会受到地域限制。
  • WARP软件是基于wireguard魔改的。
  • WARP有安卓和Windows的客户端，但是使用人数过多，体验并不好
  • Linux下通过WARP代理能实现20MB/s的下载速度。

WARP on Linux

安装配置

• 缘由：WARP下的PT做种快得多。(不是，是因为网络硬盘，所以下载多少要占用多少上传)
• 参考教程。
• 脚本和所需文件在Z:\shaojiemike\Documents\文献\计算机网络目录下。这里先使用fjw的脚本。

1. 通过注册脚本register.py，获得私钥和分配的ip
2. 配置wg。其中Endpoint端口可从官方文档中找到，默认的2408很可能被封。WARP with firewall · Cloudflare Zero Trust docs

   1 2 3 4 5 6 7 8 9 10

   [Interface] PrivateKey = xxxx #register.py的私钥 Address = xxx/32,xxx/128 #register.py的ipv4和ipv6 Table = off # off禁止wg修改路由表 [Peer] PublicKey = bmXOC+F1FxEMF9dyiK2H5/1SUtzH0JuVo51h2wPfgyo= AllowedIPs = 0.0.0.0/0,::/0 Endpoint = [2606:4700:d0::a29f:c001]:500 #2408, 1701, 500, 4500 PersistentKeepalive = 25

3. warp使用的不是标准的wg协议，root下运行，需要通过一个nft脚本main.sh修改包的3个字节。
   1. 安装nft apt-get install nftables
   2. 需要/etc/default/warp-helper文件填写对应的
   3. ROUTING_ID对应register.py的ROUTING_ID。注意三个数之间没空格
   4. UPSTREAM对应wg-conf里Endpoint。比如：

   5.   ROUTING_ID=11,45,14
        UPSTREAM=[2606:4700:d0::a29f:c001]:500
        
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      2. 最后开启路由表,Root权限运行` ip route add default dev warp proto static scope link table default`
      
      ## WARP on OpenWRT 
      
      * 目的：为了防止大量流量通过WARP，导致被官方封禁，所以只在OpenWRT上配置WARP分流github的流量。
      * 实现思路：
        * 运行python脚本，通过github的API获得所有的github域名ip，
        * 使用iptables的warp_out表，将目的地址为github域名ip路由到WARP的虚拟网卡上。
      
      ### WARP Wireguard Establishment
      
      ```bash
      python register.py #自动生成warp-op.conf,warp.conf和warp-helper
      mv warp-helper /etc/default
      # cat main.sh
      # cat warp-op.conf
      vim /etc/config/network #填写warp-op.conf内容，默认只转发172.16.0.0/24来测试连接
      ifup warp #启动warp, 代替wg-quick up warp.conf
      bash main.sh #启动防火墙实现报文头关键三字节修改
      nft list ruleset #查看防火墙，是否配置成功
      wg #查看warp状态，测试是否连接成果
      
      

这时还没创建warp_out路由表，所以还不能通过WARP出数据。

1
2
3
4
5
6
7
8

#/etc/config/network
config interface 'warp'
        option proto 'wireguard'
        option private_key 'wKpvFCOk4sf8d/RD001TF7sNQ39bWnllpqaFf8QnHG4='
        option listen_port '51825'
        list addresses '172.16.0.2/32'
        list addresses '2606:4700:110:8466:d4ea:ffb8:10da:470f/128'
        #option disabled '1' 

然后WebUI点击apply 或者命令行运行ifconfig warp down && ifup

Network planning and design

添加了WARP的网络出口后，路由器不在只是通过WAN出数据。防火墙需要更新：

• 原路返回规则。
  • 针对有公网ip的接口，需要原路返回。
    • 配置来自wan和WARP的信报，使用wan和WARP的路由表，优先级3
    • 来自wan的比如来自外部的ssh，为了防止失联。
    • 来自WARP的比如wget --bind-address=WARP_ip来模拟
  • 内网地址没有必要配置，因为通过内网地址访问host，则dst必然也是内网地址。因此会匹配main中的内网地址规则。
• wan和WARP的路由表内各自走wan和WARP的网卡
• 为了使得原本wg正常运行，10: from all lookup main suppress_prefixlength 1
  • 假如warp_out是defualt规则，该项也是为了防止失联。
• 创建warp_out的空路由表1000: from all lookup warp_out,优先级1000

1
2
3
4
5
6
7
8

root@tsjOp:~/warp# ip rule
0:      from all lookup local
3:      from 114.214.233.141/22 iif eth1 lookup wan
3:      from 172.16.0.2 iif warp lookup warp
10:     from all lookup main suppress_prefixlength 1
1000:   from all lookup warp_out
32766:  from all lookup main
32767:  from all lookup default

填充warp_out路由表

1
2
3

cd ip_route
mv ../github_ipv4.txt .
python fill_ip_table.py --table warp_out --iface warp --p2p -f github_ipv4.txt

对所有github域名的ip执行类似ip ro add 192.30.252.0/22 dev warp proto static table warp_out操作。

测试

1
2
3

mtr www.github.com
ssh -vT git@github.com
git clone https://github.com/llvm/llvm-project

添加到启动项

修改/etc/rc.local

1
2
3
4
5
6
7
8
9

# Put your custom commands here that should be executed once
# the system init finished. By default this file does nothing.


sleep 30 && cd /root/warp/ip_route && python fill_ip_table.py --table warp_out --iface warp --p2p -f github_ipv4.txt

/root/warp/main.sh #重新添加防火墙

exit 0

WARP on Windows

基于1.1.1.1 的安装windows版本直接白嫖

需要进一步的研究学习

暂无

遇到的问题

暂无

开题缘由、总结、反思、吐槽~~

参考文献

https://gist.github.com/iBug/3107fd4d5af6a4ea7bcea4a8090dcc7e

glados