]> git.feebdaed.xyz Git - 0xmirror/grpc.git/commitdiff
[Fix][Python] Increase timeout for linux distribtests jobs for release and master...
authorSreenithi Sridharan <ssreenithi@google.com>
Mon, 3 Nov 2025 23:23:41 +0000 (15:23 -0800)
committerCopybara-Service <copybara-worker@google.com>
Mon, 3 Nov 2025 23:27:12 +0000 (15:27 -0800)
The newly migrated pyproject.toml build system (in #40833) seems to have much slower build times compared to the previous setup.py builds. This difference has also been noted by few users ([Ref1](https://github.com/pypa/pip/issues/7294), [Ref2](https://zameermanji.com/blog/2021/6/14/building-wheels-with-pip-is-getting-slower/)) most likely due to the overhead of creating isolated build environments for each build.

Some new runtimes for each variant target noted from different recent runs below:
```
2025-11-01 10:19:02,937 PASSED: build_artifact.python_musllinux_1_2_x86_cp312-cp312 [time=4267.1sec, retries=0:0]
2025-11-01 10:19:05,501 PASSED: build_artifact.python_musllinux_1_2_x86_cp311-cp311 [time=4269.7sec, retries=0:0]
2025-11-01 10:19:07,152 PASSED: build_artifact.python_musllinux_1_2_x86_cp310-cp310 [time=4287.0sec, retries=0:0]

2025-11-01 11:43:22,269 PASSED: build_artifact.python_manylinux2014_x86_cp312-cp312 [time=4644.4sec, retries=0:0]
2025-11-01 11:43:37,257 PASSED: build_artifact.python_manylinux2014_x86_cp314-cp314 [time=4659.4sec, retries=0:0]
2025-11-01 11:43:37,525 PASSED: build_artifact.python_manylinux2014_x86_cp311-cp311 [time=4659.7sec, retries=0:0]
2025-11-01 11:43:40,671 PASSED: build_artifact.python_manylinux2014_x86_cp310-cp310 [time=4662.8sec, retries=0:0]
2025-11-01 11:43:51,663 PASSED: build_artifact.python_manylinux2014_x86_cp39-cp39 [time=4673.8sec, retries=0:0]
2025-11-01 11:43:51,708 PASSED: build_artifact.python_manylinux2014_x86_cp313-cp313 [time=4673.8sec, retries=0:0]

2025-11-01 04:59:11,117 PASSED: build_artifact.python_linux_extra_armv7_cp311-cp311 [time=5172.2sec, retries=0:0]
2025-11-01 04:59:31,002 PASSED: build_artifact.python_linux_extra_armv7_cp314-cp314 [time=5157.0sec, retries=0:0]
2025-11-01 04:59:53,485 PASSED: build_artifact.python_linux_extra_armv7_cp39-cp39 [time=5221.7sec, retries=0:0]
```

The builds take anywhere between 70-90 minutes for each target. We have a total of 30 targets currently and 12 targets  are built parallelly in a batch. So for about 3 batches of 12 targets each and a worst case time of 90 minutes per batch, the `build_artifact` phase itself should take about 4.5 hours to complete, followed by the remaining `package` and `distribtest` phases which should be comparatively shorter.

So, I believe about 7 hours should be enough, but just keeping an extra buffer and increasing the timeout to 8 hours.

Closes #41000

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41000 from sreenithi:increase_distribtests_timeout 44c94b0ffd83469856d9a62f8edcee8e69fa252a
PiperOrigin-RevId: 827672050

tools/internal_ci/linux/grpc_distribtests_python.cfg
tools/internal_ci/linux/release/grpc_distribtests_python.cfg

index c0344f0425b8ee839a94c64e09389a0d0429770d..c6010a2ccc5446fecebcdd78cf549acc8a737221 100644 (file)
@@ -16,7 +16,7 @@
 
 # Location of the continuous shell script in repository.
 build_file: "grpc/tools/internal_ci/linux/grpc_distribtests_python.sh"
-timeout_mins: 240
+timeout_mins: 480
 action {
   define_artifacts {
     regex: "**/*sponge_log.*"
index 9ac28d65af88a703355e738653ac67cfaf453cc0..afd8ea026ae0bec5ac69b258ab384b880441ab98 100644 (file)
@@ -16,7 +16,7 @@
 
 # Location of the continuous shell script in repository.
 build_file: "grpc/tools/internal_ci/linux/grpc_distribtests_python.sh"
-timeout_mins: 240
+timeout_mins: 480
 action {
   define_artifacts {
     regex: "**/*sponge_log.*"