[GAP Forum] Multitasking with GAP

[Christopher Jefferson]

Hi Alan,

You could look at "ParListByFork", from the IO package. This creates (using 'fork') some extra copies of GAP, and communicates with them via pipes.

There are some significant limitations of this function -- it requires the return values aren't too complicated (in practice, they must be accepted by the 'IO_Pickle' function).

Chris

-----Original Message-----
From: Alan Hylton <agh314 at lehigh.edu> 
Sent: Tuesday, May 18, 2021 6:35 AM
To: forum at gap-system.org
Subject: [GAP Forum] Multitasking with GAP

Howdy,

Suppose I have a list of lists, and I wish to run some time-consuming process on each of these sub-list (each sub-list is independent, so I am not worried about race conditions).

I think the easiest way to demonstrate my thought process is with code:

I have a list of lists and the number of cores I wish to use - master_list := [ [...], [...], ..., [...]]; n := Length(master_list); cores:=15;

I have a time consuming function whose arguments are ranges into master_list - time_consuming_func := function(start_index, stop_index) ...
end;

I portion out master_list, creating a list a tasks - task_list:=[]; start:=1; for i in [1..cores] do  flag:=0;  if i <= n mod cores then
  flag:=1;
 fi;
 if i > n then
  break;
 fi;
 len:=Int(n/cores)+flag;

 Add(task_list, RunTask( time_consuming_func , start, start+len-1));

 start:=start+len;
od;

I had several hopes:
1: I could get a list of tasks, and then use something like TaskFinished to see if each are done
2: Store the result of each time_consuming_func in the global master_list

But I ran into one problem: RunTask seems to be blocking. Instead of spawning a process and continuing with my loop, it waits until each task finishes. I considered DelayTask instead of RunTask so that I could just use WaitTask, but DelayTask does not seem to exist (similarly for asynchronous tasks). Is there an alternative? To follow the documentation, I'd like to avoid the lower-level CreateThread if I can.

Also, number 2 makes some assumptions on how memory works. Is it actually valid to have a thread working on element i of master_list modify the global master_list[i]?

I'd greatly appreciate any insight!

Thanks,
Alan
_______________________________________________
Forum mailing list
Forum at gap-system.org
https://mail.gap-system.org/mailman/listinfo/forum