test: shorten the slowest tests unless MPITEST_STRESS is set

doc/wiki/cvar.md

@@ -3,11 +3,11 @@
 ## Table of Contents
 
 - [Collective Algorithms](#collective-algorithms) (301)
-- [CH4 Device Layer](#ch4-device-layer) (96)
-- [CH3 Device Layer](#ch3-device-layer) (26)
+- [CH4 Device Layer](#ch4-device-layer) (95)
+- [CH3 Device Layer](#ch3-device-layer) (25)
 - [Process Management](#process-management) (7)
 - [Debugging and Development](#debugging-and-development) (15)
-- [Other](#other) (25)
+- [Other](#other) (26)
 
 ## Collective Algorithms
 
@@ -2807,16 +2807,6 @@ Controls whether the CH4 CMA shared-memory IPC path may be selected for intranod
 
 Used in functions: `MPIDI_CMA_init_local`, `MPIDI_CMA_get_ipc_attr`
 
-### MPIR_CVAR_CH4_COMM_CONNECT_TIMEOUT
-
-Sets the default timeout period for CH4 dynamic-process connection attempts. It is used by the root process in `MPI_Comm_connect` when connecting to a server communicator whose named port exists but has no pending accept, and the per-call MPI info key `timeout` overrides this CVAR when provided. It does not rely on another CVAR being enabled. When active, this CVAR sets how long the connecting side waits for the accept-side handshake before returning a port error.
-
-* 180 - default, wait up to 180 seconds for the accept-side handshake.
-* positive integer - wait up to the specified number of seconds for the accept-side handshake.
-* 0 or negative integer - do not continue waiting after the initial progress check.
-
-Used in functions: `MPID_Comm_connect`, `dynamic_intercomm_create`, `establish_peer_conn`
-
 ### MPIR_CVAR_CH4_ENABLE_STREAM_WORKQ
 
 Controls whether CH4 stream enqueue operations use the stream work queue path. It is used by send, receive, nonblocking send, nonblocking receive, wait, and waitall enqueue operations; when disabled, those operations call the regular enqueue implementation instead. When enabled, the communicator or enqueue request must be associated with an MPIX GPU stream and that stream must provide the work queue, with progress supplied by the corresponding stream progress thread. When active, this CVAR enables deferring stream enqueue operations through the stream work queue using GPU trigger and completion events.
@@ -3726,16 +3716,6 @@ Used in functions: `MPIDI_UCX_recv`
 
 ## CH3 Device Layer
 
-### MPIR_CVAR_CH3_COMM_CONNECT_TIMEOUT
-
-Sets the default timeout period for CH3 dynamic-process connection attempts. It is used by the root process in `MPI_Comm_connect` when connecting to a server communicator whose named port exists but has no pending accept, and the per-call MPI info key `timeout` overrides this CVAR when provided. It does not rely on another CVAR being enabled, but it is active only for CH3 builds with dynamic-process support. When active, this CVAR sets how long the connecting side waits for the accept-side handshake before revoking the connection request and returning a port error.
-
-* 180 - default, wait up to 180 seconds for the accept-side handshake.
-* positive integer - wait up to the specified number of seconds for the accept-side handshake.
-* 0 or negative integer - do not continue waiting after the initial progress check.
-
-Used in functions: `MPIDI_Comm_connect`, `MPIDI_Create_inter_root_communicator_connect`
-
 ### MPIR_CVAR_CH3_EAGER_MAX_MSG_SIZE
 
 Sets the eager-send message-size threshold for CH3 virtual connections. It is used when a CH3 virtual connection is initialized, after the ready-send eager threshold is set to no limit and before channel-specific virtual-connection initialization runs. It does not rely on another CVAR or mode being active. When active, this CVAR sets the maximum CH3 message size sent eagerly before switching to rendezvous mode.
@@ -4229,6 +4209,18 @@ Controls the line width used when formatting MPI error stack output. It is used
 
 Used in functions: `MPIR_Err_stack_init`, `MPIR_Err_print_stack_string`
 
+### MPIR_CVAR_COMM_CONNECT_TIMEOUT
+
+Sets the default timeout period for dynamic-process connection attempts. It is used by the root process in `MPI_Comm_connect` when connecting to a server communicator whose named port exists but has no pending accept, and the per-call MPI info key `timeout` overrides this CVAR when provided. It does not rely on another CVAR being enabled, and both the CH3 and CH4 devices use it. When active, this CVAR sets how long the connecting side waits for the accept-side handshake before returning a port error.
+
+* 180 - default, wait up to 180 seconds for the accept-side handshake.
+* positive integer - wait up to the specified number of seconds for the accept-side handshake.
+* 0 or negative integer - do not continue waiting after the initial progress check.
+
+Alternate names: MPIR_CVAR_CH3_COMM_CONNECT_TIMEOUT, MPIR_CVAR_CH4_COMM_CONNECT_TIMEOUT
+
+Used in functions: `MPID_Comm_connect`, `dynamic_intercomm_create`, `establish_peer_conn`, `MPIDI_Comm_connect`, `MPIDI_Create_inter_root_communicator_connect`
+
 ### MPIR_CVAR_COMM_SPLIT_USE_QSORT
 
 Controls whether `MPI_Comm_split` uses the qsort-based sorting path when ordering processes by key. It is used while constructing the rank order for a new communicator after the split color has selected participating local processes, and for intercommunicators it is also used when ordering the matching remote group. This path requires qsort support to be available at build time; otherwise the insertion-sort path is used. When active, this CVAR enables qsort-based stable ordering of split participants by key.

src/mpi/spawn/spawn_impl.c

@@ -21,6 +21,27 @@
 #define SOCKET_EINTR        EINTR
 #endif
 
+/*
+=== BEGIN_MPI_T_CVAR_INFO_BLOCK ===
+
+cvars:
+    - name        : MPIR_CVAR_COMM_CONNECT_TIMEOUT
+      category    : COMMUNICATOR
+      alt-env     : MPIR_CVAR_CH3_COMM_CONNECT_TIMEOUT, MPIR_CVAR_CH4_COMM_CONNECT_TIMEOUT
+      type        : int
+      default     : 180
+      class       : none
+      verbosity   : MPI_T_VERBOSITY_USER_BASIC
+      scope       : MPI_T_SCOPE_GROUP_EQ
+      description : >-
+        The default time out period in seconds for a connection attempt to the
+        server communicator where the named port exists but no pending accept.
+        User can change the value for a specified connection through its info
+        argument.
+
+=== END_MPI_T_CVAR_INFO_BLOCK ===
+*/
+
 int MPIR_Comm_get_parent_impl(MPI_Comm * parent)
 {
     *parent = (MPIR_Process.comm_parent == NULL) ? MPI_COMM_NULL :

src/mpid/ch3/src/ch3u_port.c

@@ -6,26 +6,6 @@
 #include "mpidi_ch3_impl.h"
 #include "mpid_port.h"
 
-/*
-=== BEGIN_MPI_T_CVAR_INFO_BLOCK ===
-
-cvars:
-    - name        : MPIR_CVAR_CH3_COMM_CONNECT_TIMEOUT
-      category    : CH3
-      type        : int
-      default     : 180
-      class       : none
-      verbosity   : MPI_T_VERBOSITY_USER_BASIC
-      scope       : MPI_T_SCOPE_GROUP_EQ
-      description : >-
-        The default time out period in seconds for a connection attempt to the
-        server communicator where the named port exists but no pending accept.
-        User can change the value for a specified connection through its info
-        argument.
-
-=== END_MPI_T_CVAR_INFO_BLOCK ===
-*/
-
 /*
  * This file replaces ch3u_comm_connect.c and ch3u_comm_accept.c .  These
  * routines need to be used together, particularly since they must exchange
@@ -600,7 +580,7 @@ int MPIDI_Comm_connect(const char *port_name, MPIR_Info *info, int root,
 
     if (rank == root)
     {
-        int timeout = MPIR_CVAR_CH3_COMM_CONNECT_TIMEOUT;
+        int timeout = MPIR_CVAR_COMM_CONNECT_TIMEOUT;
 
         /* Check if user specifies timeout threshold. */
         if (info != NULL) {

src/mpid/ch4/src/ch4_spawn.c

@@ -7,26 +7,6 @@
 
 #define MPIDI_DYNPROC_NAME_MAX MPID_MAX_BC_SIZE
 
-/*
-=== BEGIN_MPI_T_CVAR_INFO_BLOCK ===
-
-cvars:
-    - name        : MPIR_CVAR_CH4_COMM_CONNECT_TIMEOUT
-      category    : CH4
-      type        : int
-      default     : 180
-      class       : none
-      verbosity   : MPI_T_VERBOSITY_USER_BASIC
-      scope       : MPI_T_SCOPE_GROUP_EQ
-      description : >-
-        The default time out period in seconds for a connection attempt to the
-        server communicator where the named port exists but no pending accept.
-        User can change the value for a specified connection through its info
-        argument.
-
-=== END_MPI_T_CVAR_INFO_BLOCK ===
-*/
-
 int MPID_Comm_spawn_multiple(int count, char *commands[], char **argvs[], const int maxprocs[],
                              MPIR_Info * info_ptrs[], int root, MPIR_Comm * comm_ptr,
                              MPIR_Comm ** intercomm, int errcodes[])
@@ -439,7 +419,7 @@ int MPID_Comm_connect(const char *port_name, MPIR_Info * info, int root, MPIR_Co
     int mpi_errno = MPI_SUCCESS;
     MPIR_FUNC_ENTER;
 
-    int timeout = MPIR_CVAR_CH4_COMM_CONNECT_TIMEOUT;
+    int timeout = MPIR_CVAR_COMM_CONNECT_TIMEOUT;
     if (info != NULL) {
         int info_flag = 0;
         char info_value[MPI_MAX_INFO_VAL + 1];

test/mpi/errors/spawn/connect_timeout.c

@@ -12,7 +12,12 @@
 #define IF_VERBOSE(cond, a) if (verbose && (cond)) { printf a ; fflush(stdout); }
 #define SERVER_GID 0
 #define CLIENT_GID 1
-#define TIMEOUT "5"
+/* Connect timeouts in seconds: the default (set through the CVAR) and the one
+ * passed in the info argument. MPITEST_STRESS selects the longer values. */
+#define DEFAULT_TIMEOUT 3
+#define INFO_TIMEOUT "1"
+#define DEFAULT_TIMEOUT_STRESS 20
+#define INFO_TIMEOUT_STRESS "5"
 
 /* This test checks whether a MPI_COMM_CONNECT can eventually time out and return
  * an error of class MPI_ERR_PORT when the named port exists but no pending
@@ -45,6 +50,7 @@
 
 int rank, nproc;
 int verbose = 0;
+const char *info_timeout;
 
 int test_mismatched_accept(MPI_Comm intra_comm, int gid);
 int test_no_accept(MPI_Comm intra_comm, int gid);
@@ -143,7 +149,7 @@ int test_mismatched_accept(MPI_Comm intra_comm, int gid)
             errs += check_errno(mpi_errno2, MPI_SUCCESS);
         }
     } else if (gid == SERVER_GID) {
-        /* NOTE: if accept hangs, try increase MPIR_CVAR_CH3_COMM_CONN_TIMEOUT. */
+        /* NOTE: if accept hangs, try increase MPIR_CVAR_COMM_CONNECT_TIMEOUT. */
         IF_VERBOSE(intra_rank == 0, ("server: accepting connection to <%s>.\n", port));
         mpi_errno = MPI_Comm_accept(port, MPI_INFO_NULL, 0, intra_comm, &comm);
         if (mpi_errno == MPI_SUCCESS) {
@@ -187,11 +193,11 @@ int test_no_accept(MPI_Comm intra_comm, int gid)
         errs += check_errno(mpi_errno, MPI_ERR_PORT);
 
         MPI_Info_create(&info);
-        MPI_Info_set(info, "timeout", TIMEOUT);
+        MPI_Info_set(info, "timeout", info_timeout);
 
         IF_VERBOSE(intra_rank == 0,
                    ("client: case 2 - connecting to <%s> with specified timeout <%s>s.\n", port,
-                    TIMEOUT));
+                    info_timeout));
         mpi_errno = MPI_Comm_connect(port, info, 0, intra_comm, &comm);
         errs += check_errno(mpi_errno, MPI_ERR_PORT);
 
@@ -209,6 +215,21 @@ int test_no_accept(MPI_Comm intra_comm, int gid)
     return errs;
 }
 
+/* Set the default connect timeout through MPICH's CVAR. Other MPI
+ * implementations keep their own default. */
+static void set_default_timeout(int timeout)
+{
+    int provided, cvar_index, count;
+    MPI_T_cvar_handle handle;
+
+    MPI_T_init_thread(MPI_THREAD_SINGLE, &provided);
+    if (MPI_T_cvar_get_index("MPIR_CVAR_COMM_CONNECT_TIMEOUT", &cvar_index) == MPI_SUCCESS) {
+        MPI_T_cvar_handle_alloc(cvar_index, NULL, &handle, &count);
+        MPI_T_cvar_write(handle, &timeout);
+        MPI_T_cvar_handle_free(&handle);
+    }
+    MPI_T_finalize();
+}
 
 int main(int argc, char *argv[])
 {
@@ -223,6 +244,14 @@ int main(int argc, char *argv[])
     MPI_Comm_size(MPI_COMM_WORLD, &nproc);
     MPI_Comm_rank(MPI_COMM_WORLD, &rank);
 
+    if (MTestGetStressLevel()) {
+        set_default_timeout(DEFAULT_TIMEOUT_STRESS);
+        info_timeout = INFO_TIMEOUT_STRESS;
+    } else {
+        set_default_timeout(DEFAULT_TIMEOUT);
+        info_timeout = INFO_TIMEOUT;
+    }
+
     if (nproc < 2) {
         printf("At least two processes needed to run this test.\n");
         goto exit;

test/mpi/errors/spawn/testlist.in

@@ -1,7 +1,7 @@
 badport 2
 @namepub_tests@unpub 1
 @namepub_tests@lookup_name 1
-connect_timeout_no_accept 2 env=MPIR_CVAR_CH3_COMM_CONNECT_TIMEOUT=20 env=MPIR_CVAR_CH4_COMM_CONNECT_TIMEOUT=20
-connect_timeout_mismatch 2 env=MPIR_CVAR_CH3_COMM_CONNECT_TIMEOUT=20 env=MPIR_CVAR_CH4_COMM_CONNECT_TIMEOUT=20
-connect_timeout_no_accept 5 env=MPIR_CVAR_CH3_COMM_CONNECT_TIMEOUT=20 env=MPIR_CVAR_CH4_COMM_CONNECT_TIMEOUT=20
-connect_timeout_mismatch 5 env=MPIR_CVAR_CH3_COMM_CONNECT_TIMEOUT=20 env=MPIR_CVAR_CH4_COMM_CONNECT_TIMEOUT=20
+connect_timeout_no_accept 2
+connect_timeout_mismatch 2
+connect_timeout_no_accept 5
+connect_timeout_mismatch 5

test/mpi/maint/all_mpitests.txt

@@ -499,6 +499,7 @@ mpi_t/cvarwrite
 mpi_t/getindex
 
 part/start_pready
+part/pingping
 part/parrived
 part/multipart
 part/nonblocking_pready

test/mpi/part/pingping.c

@@ -4,13 +4,15 @@
  */
 
 #include "mpitest.h"
-#include "mpi.h"
-#include <stdio.h>
-#include <stdlib.h>
-#include "mtest_dtp.h"
 #include "dtpools.h"
+#include "mtest_dtp.h"
 #include <assert.h>
 
+#ifdef MULTI_TESTS
+#define run part_pingping
+int run(const char *arg);
+#endif
+
 /* Extended from src/pt2pt/pingping.c with partitioned APIs. */
 
 /* the stress testing of unexpected message queue is covered by the pt2pt/pingping,
@@ -24,7 +26,8 @@ static char MTEST_Descrip[] = "Pingping test with partitioned point-to-point rou
 */
 
 #define MAX_TOTAL_MSG_SIZE (32 * 1024 * 1024)
-#define MAXMSG (4096)
+#define MAXMSG (16)
+#define MAXMSG_STRESS (4096)
 
 /* Used on both sender and receiver sides. If the total count of datatype elements
  * cannot be completely divided, we set local partitions to 1. */
@@ -71,47 +74,44 @@ static void recv_test(void *rbuf, int partitions, MPI_Count count, MPI_Datatype
     MPI_Request_free(&req);
 }
 
-int main(int argc, char *argv[])
+static int pingping(int seed, int testsize, int sendcnt, int recvcnt,
+                    const char *basic_type, mtest_mem_type_e sendmem, mtest_mem_type_e recvmem)
 {
     int errs = 0, err;
     int rank, size, source, dest;
     int minsize = 2, nmsg, maxmsg;
-    int seed, testsize;
-    MPI_Aint sendcnt, recvcnt;
     MPI_Comm comm;
     DTP_pool_s dtp;
     struct mtest_obj send, recv;
-    mtest_mem_type_e sendmem, recvmem;
-    char *basic_type;
 
-    MTest_Init(&argc, &argv);
+    int world_rank;
+    MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);
 
-    MTestArgList *head = MTestArgListCreate(argc, argv);
-    seed = MTestArgListGetInt(head, "seed");
-    testsize = MTestArgListGetInt(head, "testsize");
-    sendcnt = MTestArgListGetLong(head, "sendcnt");
-    recvcnt = MTestArgListGetLong(head, "recvcnt");
-    basic_type = MTestArgListGetString(head, "type");
-    sendmem = MTestArgListGetMemType(head, "sendmem");
-    recvmem = MTestArgListGetMemType(head, "recvmem");
+    static char test_desc[200];
+    snprintf(test_desc, 200,
+             "./pingping -seed=%d -testsize=%d -type=%s -sendcnt=%d -recvcnt=%d -sendmem=%s -recvmem=%s",
+             seed, testsize, basic_type, sendcnt, recvcnt, MTest_memtype_name(sendmem),
+             MTest_memtype_name(recvmem));
+    if (world_rank == 0) {
+        MTestPrintfMsg(1, " %s\n", test_desc);
+    }
 
     err = DTP_pool_create(basic_type, sendcnt, seed, &dtp);
     if (err != DTP_SUCCESS) {
-        fprintf(stderr, "Error while creating send pool (%s,%ld)\n", basic_type, sendcnt);
+        fprintf(stderr, "Error while creating send pool (%s,%d)\n", basic_type, sendcnt);
         fflush(stderr);
     }
 
-    MTestArgListDestroy(head);
-
     MTest_dtp_obj_start(&send, "send", dtp, sendmem, 0, false);
     MTest_dtp_obj_start(&recv, "recv", dtp, recvmem, 0, false);
 
     int nbytes;
     MPI_Type_size(dtp.DTP_base_type, &nbytes);
     nbytes *= sendcnt;
     maxmsg = MAX_TOTAL_MSG_SIZE / nbytes;
-    if (maxmsg > MAXMSG)
-        maxmsg = MAXMSG;
+    int max_nmsg = MTestGetStressLevel()? MAXMSG_STRESS : MAXMSG;
+    if (maxmsg > max_nmsg)
+        maxmsg = max_nmsg;
 
     /* The following illustrates the use of the routines to
      * run through a selection of communicators and datatypes.
@@ -197,7 +197,21 @@ int main(int argc, char *argv[])
     MTest_dtp_obj_finish(&send);
     MTest_dtp_obj_finish(&recv);
     DTP_pool_free(dtp);
+    return errs;
+}
+
+int run(const char *arg)
+{
+    int errs = 0;
+
+    struct dtp_args dtp_args;
+    dtp_args_init_arg(&dtp_args, MTEST_DTP_PT2PT, arg);
+    while (dtp_args_get_next(&dtp_args)) {
+        errs += pingping(dtp_args.seed, dtp_args.testsize,
+                         dtp_args.count, dtp_args.u.pt2pt.recvcnt,
+                         dtp_args.basic_type, dtp_args.u.pt2pt.sendmem, dtp_args.u.pt2pt.recvmem);
+    }
+    dtp_args_finalize(&dtp_args);
 
-    MTest_Finalize(errs);
-    return MTestReturnValue(errs);
+    return errs;
 }