Skip to content

Commit af3a497

Browse files
lawrence-forooghianlawrence-forooghian
committed
ECO-4787: Add interception proxy prototype
- start-interception-proxy adapted from https://github.com/ably/sdk-test-proxy at 82e93a7 Some TODOs which aren’t really important right now because this is just a prototype: - TODO fix type checking for interception proxy — `npm run build` does it properly, but tried to reproduce the way we do it for modulereport and it didn’t work - TODO fix linting for interception proxy — doesn’t seem to be catching lint errors - TODO linting / type checking etc for Python code Also: > Add test:playwright:open-browser script > > Lets you open a headed browser which is configured to use the > interception proxy. Useful for local debugging of browser tests.
1 parent c4e9703 commit af3a497

25 files changed

+2169
-113
lines changed

‎.github/workflows/test-browser.yml‎

Lines changed: 59 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,11 +25,69 @@ jobs:
2525
with:
2626
node-version: 20.x
2727
- run: npm ci
28+
29+
# Set up Python (for pipx)
30+
- uses: actions/setup-python@v5
31+
with:
32+
python-version: '3.12'
33+
34+
# Install pipx (for mitmproxy)
35+
# See https://pipx.pypa.io/stable/installation/
36+
- name: Install pipx
37+
run: |
38+
python3 -m pip install --user pipx
39+
sudo pipx --global ensurepath
40+
41+
# https://docs.mitmproxy.org/stable/overview-installation/#installation-from-the-python-package-index-pypi
42+
- name: Install mitmproxy
43+
run: |
44+
pipx install mitmproxy
45+
# We use this library in our addon
46+
pipx inject mitmproxy websockets
47+
48+
- name: Generate mitmproxy SSL certs
49+
run: mitmdump -s test/mitmproxy_addon_generate_certs_and_exit.py
50+
51+
- name: Start interception proxy server
52+
run: ./start-interception-proxy
53+
2854
- name: Install Playwright browsers and dependencies
2955
run: npx playwright install --with-deps
30-
- env:
56+
57+
# For certutil
58+
- name: Install NSS tools
59+
run: sudo apt install libnss3-tools
60+
61+
# This is for Chromium (see https://chromium.googlesource.com/chromium/src/+/master/docs/linux/cert_management.md)
62+
# Note this is the same command that we use for adding it to the Firefox profile (see playwrightHelpers.js)
63+
- name: Install mitmproxy root CA in NSS shared DB
64+
run: |
65+
mkdir -p ~/.pki/nssdb
66+
certutil -A -d sql:$HOME/.pki/nssdb -t "C" -n "Mitmproxy Root Cert" -i ~/.mitmproxy/mitmproxy-ca-cert.pem
67+
certutil -L -d sql:$HOME/.pki/nssdb
68+
69+
# This is for WebKit (I think because it uses OpenSSL)
70+
- name: Install mitmproxy root CA in /usr/local/share/ca-certificates
71+
run: |
72+
sudo cp ~/.mitmproxy/mitmproxy-ca-cert.cer /usr/local/share/ca-certificates/mitmproxy-ca-cert.crt
73+
sudo update-ca-certificates
74+
75+
- name: Run the tests
76+
env:
3177
PLAYWRIGHT_BROWSER: ${{ matrix.browser }}
3278
run: npm run test:playwright
79+
80+
- name: Save interception proxy server logs
81+
if: always()
82+
run: sudo journalctl -u ably-sdk-test-proxy.service > interception-proxy-logs.txt
83+
84+
- name: Upload interception proxy server logs
85+
if: always()
86+
uses: actions/upload-artifact@v4
87+
with:
88+
name: interception-proxy-logs-${{ matrix.browser }}
89+
path: interception-proxy-logs.txt
90+
3391
- name: Upload test results
3492
if: always()
3593
uses: ably/test-observability-action@v1

‎.github/workflows/test-node.yml‎

Lines changed: 112 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,9 +25,120 @@ jobs:
2525
with:
2626
node-version: ${{ matrix.node-version }}
2727
- run: npm ci
28-
- run: npm run test:node
28+
29+
# Set up Python (for pipx)
30+
- uses: actions/setup-python@v5
31+
with:
32+
python-version: '3.12'
33+
34+
# Install pipx (for mitmproxy)
35+
# See https://pipx.pypa.io/stable/installation/
36+
- name: Install pipx
37+
run: |
38+
python3 -m pip install --user pipx
39+
sudo pipx --global ensurepath
40+
41+
# https://docs.mitmproxy.org/stable/overview-installation/#installation-from-the-python-package-index-pypi
42+
- name: Install mitmproxy
43+
run: |
44+
pipx install mitmproxy
45+
# We use this library in our addon
46+
pipx inject mitmproxy websockets
47+
48+
- name: Create a user to run the tests
49+
run: sudo useradd --create-home ably-test-user
50+
51+
- name: Create a group for sharing the working directory
52+
run: |
53+
sudo groupadd ably-test-users
54+
# Add relevant users to the group
55+
sudo usermod --append --groups ably-test-users $USER
56+
sudo usermod --append --groups ably-test-users ably-test-user
57+
# Give the group ownership of the working directory and everything under it...
58+
sudo chown -R :ably-test-users .
59+
# ...and give group members full read/write access to its contents (i.e. rw access to files, rwx access to directories)
60+
# (We use xargs because `find` does not fail if an `exec` command fails; see https://serverfault.com/a/905039)
61+
find . -type f -print0 | xargs -n1 -0 chmod g+rw
62+
find . -type d -print0 | xargs -n1 -0 chmod g+rwx
63+
# TODO understand better
64+
#
65+
# This is to make `npm run` work when run as ably-test-user; else it fails because of a `statx()` call on package.json:
66+
#
67+
# > 2024-04-17T13:08:09.1302251Z [pid 2051] statx(AT_FDCWD, `"/home/runner/work/ably-js/ably-js/package.json"`, AT_STATX_SYNC_AS_STAT, STATX_ALL, 0x7f4875ffcb40) = -1 EACCES (Permission denied)
68+
#
69+
# statx documentation says:
70+
#
71+
# > in the case of **statx**() with a pathname, execute (search) permission is required on all of the directories in _pathname_ that lead to the file.
72+
#
73+
# The fact that I’m having to do this probably means that I’m doing something inappropriate elsewhere. (And I don’t know what the other consequences of doing this might be.)
74+
chmod o+x ~
75+
76+
# TODO set umask appropriately, so that new files created are readable/writable by the group
77+
78+
- name: Generate mitmproxy SSL certs
79+
run: mitmdump -s test/mitmproxy_addon_generate_certs_and_exit.py
80+
81+
- name: Set up iptables rules
82+
run: |
83+
# The rules suggested by mitmproxy etc are aimed at intercepting _all_ the outgoing traffic on a machine. I don’t want that, given that we want to be able to run this test suite on developers’ machines in a non-invasive manner. Instead we just want to target traffic generated by the process that contains the Ably SDK, which we’ll make identifable by iptables by running that process as a specific user created for that purpose (ably-test-user).
84+
#
85+
# Relevant parts of iptables documentation:
86+
#
87+
# nat:
88+
# > This table is consulted when a packet that creates a new connection is encountered. It consists of three built-ins: PREROUTING (for altering packets as soon as they come in), OUTPUT (for altering locally-generated packets before routing), and POSTROUTING (for altering packets as they are about to go out).
89+
#
90+
# owner:
91+
# > This module attempts to match various characteristics of the packet creator, for locally-generated packets. It is only valid in the OUTPUT chain, and even this some packets (such as ICMP ping responses) may have no owner, and hence never match.
92+
#
93+
# REDIRECT:
94+
# > This target is only valid in the nat table, in the PREROUTING and OUTPUT chains, and user-defined chains which are only called from those chains. It redirects the packet to the machine itself by changing the destination IP to the primary address of the incoming interface (locally-generated packets are mapped to the 127.0.0.1 address). It takes one option:
95+
# >
96+
# > --to-ports port[-port]
97+
# > This specifies a destination port or range of ports to use: without this, the destination port is never altered. This is only valid if the rule also specifies -p tcp or -p udp.
98+
#
99+
# I don’t exactly understand what the nat table means; I assume its rules apply to all _subsequent_ packets in the connection, too?
100+
#
101+
# So, what I expect to happen:
102+
#
103+
# 1. iptables rule causes default-port HTTP(S) datagram from test process to get its destination IP rewritten to 127.0.0.1, and rewrites the TCP header’s destination port to 8080
104+
# 2. 127.0.0.1 destination causes OS’s routing to send this datagram on the loopback interface
105+
# 3. nature of the loopback interface means that this datagram is then received on the loopback interface
106+
# 4. mitmproxy, listening on port 8080 (not sure how or why it uses a single port for both non-TLS and TLS traffic) receives these datagrams, and uses Host header or SNI to figure out where they were originally destined.
107+
#
108+
# TODO (how) do we achieve the below on macOS? I have a feeling that it’s currently just working by accident; e.g. it's because the TCP connection to the control server exists before we start mitmproxy and hence the connection doesn’t get passed to its NETransparentProxyProvider or something. To be on the safe side, though, I’ve added a check in the mitmproxy addon so that we only mess with stuff for ports 80 or 443
109+
#
110+
# Note that in the current setup with ably-js, the test suite and the Ably SDK run in the same process. We want to make sure that we don’t intercept the test suite’s WebSocket communications with the interception proxy’s control API (which it serves at 127.0.0.1:8001), hence only targeting the default HTTP(S) ports. (TODO consider that Realtime team also run a Realtime on non-default ports when testing locally)
111+
sudo iptables --table nat --append OUTPUT --match owner --uid-owner ably-test-user --protocol tcp --destination-port 80 --jump REDIRECT --to-ports 8080
112+
sudo iptables --table nat --append OUTPUT --match owner --uid-owner ably-test-user --protocol tcp --destination-port 443 --jump REDIRECT --to-ports 8080
113+
sudo ip6tables --table nat --append OUTPUT --match owner --uid-owner ably-test-user --protocol tcp --destination-port 80 --jump REDIRECT --to-ports 8080
114+
sudo ip6tables --table nat --append OUTPUT --match owner --uid-owner ably-test-user --protocol tcp --destination-port 443 --jump REDIRECT --to-ports 8080
115+
116+
# TODO how will this behave with:
117+
#
118+
# 1. the WebSocket connection from test suite to control API (see above note; not a problem in this CI setup, think about it on macOS)
119+
# 2. the WebSocket connection from mitmproxy to control API (not an issue on Linux or macOS with our current setup since we don’t intercept any traffic from mitmproxy)
120+
# 3. the WebSocket connections that mitmproxy proxies to the interception proxy (which it sends to localhost:8002) (ditto 2)
121+
# 4. the WebSocket connections for which interception proxy is a client (not an issue for Linux or macOS with our current setup since we don’t intercept any traffic from interception proxy)
122+
123+
- name: Start interception proxy server
124+
run: ./start-interception-proxy
125+
126+
- name: Run the tests
127+
run: sudo -u ably-test-user NODE_EXTRA_CA_CERTS=~/.mitmproxy/mitmproxy-ca-cert.pem npm run test:node
29128
env:
30129
CI: true
130+
131+
- name: Save interception proxy server logs
132+
if: always()
133+
run: sudo journalctl -u ably-sdk-test-proxy.service > interception-proxy-logs.txt
134+
135+
- name: Upload interception proxy server logs
136+
if: always()
137+
uses: actions/upload-artifact@v4
138+
with:
139+
name: interception-proxy-logs-${{ matrix.node-version }}
140+
path: interception-proxy-logs.txt
141+
31142
- name: Upload test results
32143
if: always()
33144
uses: ably/test-observability-action@v1

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ react/
99
typedoc/generated/
1010
junit/
1111
test/support/mocha_junit_reporter/build/
12+
tmp/
1213

1314
# Python stuff (for interception proxy)
1415
__pycache__

‎package.json‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,8 +140,10 @@
140140
"test:node:skip-build": "mocha",
141141
"test:webserver": "grunt test:webserver",
142142
"test:playwright": "node test/support/runPlaywrightTests.js",
143+
"test:playwright:open-browser": "node test/support/openPlaywrightBrowser.js",
143144
"test:react": "vitest run",
144145
"test:package": "grunt test:package",
146+
"test:proxy": "npm run build && esr test/interception-proxy/server.ts",
145147
"concat": "grunt concat",
146148
"build": "grunt build:all && npm run build:react",
147149
"build:node": "grunt build:node",

‎start-interception-proxy‎

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
#!/usr/bin/env bash
2+
3+
# Runs the server as a background service, exiting once the server is ready to receive requests.
4+
#
5+
# Intended for use in SDKs’ CI jobs. Must be run from the root of this repository.
6+
7+
set -e
8+
9+
# We run as the current user so that we can access the generated server certificate (in ~/.mitmproxy) without having to worry about permissions. TODO consider instead generating our own cert and telling mitmproxy to use it, then we also won’t have to have that step where we run mitmproxy once just to generate the certs
10+
start_systemd_service () {
11+
systemd_service=$(cat <<SYSTEMD_SERVICE
12+
[Unit]
13+
Description=Ably SDK Test Proxy
14+
15+
[Service]
16+
WorkingDirectory=$(pwd)
17+
ExecStart=npm run test:proxy
18+
User=$USER
19+
20+
[Install]
21+
WantedBy=multi-user.target
22+
SYSTEMD_SERVICE
23+
)
24+
25+
# https://stackoverflow.com/questions/84882/sudo-echo-something-etc-privilegedfile-doesnt-work
26+
echo "${systemd_service}" | sudo tee /etc/systemd/system/ably-sdk-test-proxy.service 1>/dev/null
27+
28+
echo "Starting ably-sdk-test-proxy systemd service..." 1>&2
29+
sudo systemctl start ably-sdk-test-proxy.service
30+
echo "Started ably-sdk-test-proxy systemd service." 1>&2
31+
}
32+
33+
start_launchd_daemon () {
34+
launchd_daemon=$(cat <<LAUNCHD_DAEMON
35+
<?xml version="1.0" encoding="UTF-8"?>
36+
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
37+
<plist version="1.0">
38+
<dict>
39+
<key>Label</key>
40+
<string>com.ably.test.proxy</string>
41+
<key>WorkingDirectory</key>
42+
<string>$(pwd)</string>
43+
<key>ProgramArguments</key>
44+
<array>
45+
<string>npm</string>
46+
<string>run</string>
47+
<string>test:proxy</string>
48+
</array>
49+
<key>RunAtLoad</key>
50+
<true/>
51+
</dict>
52+
</plist>
53+
LAUNCHD_DAEMON
54+
)
55+
56+
# https://stackoverflow.com/questions/84882/sudo-echo-something-etc-privilegedfile-doesnt-work
57+
echo "${launchd_daemon}" | sudo tee /Library/LaunchDaemons/com.ably.test.proxy.plist
58+
59+
echo "Loading ably-sdk-test-proxy launchd daemon..." 1>&2
60+
sudo launchctl load /Library/LaunchDaemons/com.ably.test.proxy.plist
61+
echo "Loaded ably-sdk-test-proxy launchd daemon." 1>&2
62+
}
63+
64+
check_daemon_still_running () {
65+
if uname | grep Linux 1>/dev/null
66+
then
67+
systemctl is-active --quiet ably-sdk-test-proxy.service
68+
elif uname | grep Darwin 1>/dev/null
69+
then
70+
launchctl print system/com.ably.test.proxy 1>/dev/null
71+
fi
72+
}
73+
74+
if uname | grep Linux 1>/dev/null
75+
then
76+
start_systemd_service
77+
elif uname | grep Darwin 1>/dev/null
78+
then
79+
start_launchd_daemon
80+
else
81+
echo "Unsupported system $(uname); exiting" 1>&2
82+
exit 1
83+
fi
84+
85+
echo "Waiting for sdk-test-proxy server to start on port 8001..." 1>&2
86+
87+
# https://stackoverflow.com/questions/27599839/how-to-wait-for-an-open-port-with-netcat
88+
while ! nc -z localhost 8001; do
89+
# Check that the service hasn’t failed (else we’ll be waiting forever)
90+
check_daemon_still_running
91+
sleep 0.5
92+
done
93+
94+
echo "sdk-test-proxy server is now listening on port 8001." 1>&2

‎test/common/globals/named_dependencies.js‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,5 +18,9 @@ define(function () {
1818
async: { browser: 'node_modules/async/lib/async' },
1919
chai: { browser: 'node_modules/chai/chai', node: 'node_modules/chai/chai' },
2020
ulid: { browser: 'node_modules/ulid/dist/index.umd', node: 'node_modules/ulid/dist/index.umd' },
21+
interception_proxy_client: {
22+
browser: 'test/common/modules/interception_proxy_client',
23+
node: 'test/common/modules/interception_proxy_client',
24+
},
2125
});
2226
});

0 commit comments

Comments
 (0)