Dear GAP forum,

Peter F. Blanchard wrote [in reference to my AutAg share package]:
>> Here is my question: can someone tell me how to multiply cosets
>> of subgroups of automorphism groups? It seems that this was left out, but
>> it is probably not difficult to do. An example follows which illustrates
>> how the problem comes up when trying to calculate an intersection of
>> subgroups of an automorphism group.

I think I need an explanation from someone who knows more about the parsing
of GAP commands. I'm replying to the list because this may be of interest
to others who are interested in implementing their own group element
records...

I've repeated Peter's example here (with a few more commands added):

----------------------------------------------------------------------
gap> G := SpecialAgGroup(DihedralGroup(AgWords,8));
Group( s, d1, d2 )
gap> G.name := "G";;
gap> A := AutGroupSagGroup(G);
Group( Aut(G, [ s*d1, d1, d2 ]), InnerAut(G, d1), InnerAut(G, s) )
gap> A.name := "A";;
gap> t1 := Subgroup(A, [A.1]);
Subgroup( A, [ Aut(G, [ s*d1, d1, d2 ]) ] )
gap> t2 := Subgroup(A, [A.2]);
Subgroup( A, [ InnerAut(G, d1) ] )
gap> Intersection(t1, t2);
Error, product of <a> and <b> is not defined in
<rec1> * <rec2> called from
arg[1].operations.Stabilizer( arg[1], arg[2], arg[3] ) called from
Stabilizer( H, Coset( G ), OnRight ) called from
D.operations.Intersection( I, D ) called from
Intersection( t1, t2 ) called from
main loop
brk> a;
(Subgroup( A, [ Aut(G, [ s*d1, d1, d2 ]) ] )*InnerAut(G, IdAgWord))
brk> IsRightCoset(a);
true
brk> a.operations;
DomainOps
brk> a.operations.\*;
function ( C, D ) ... end
brk> Print(a.operations.\*);
function ( C, D )
    local  E;
    if IsRightCoset( C ) and D in Parent( C.group )  then
        E := RightCoset( C.group, C.representative * D );
    elif IsRightCoset( C )  then
        E := Elements( C ) * D;
    elif IsRightCoset( D )  then
        E := C * Elements( D );
    else
        Error( "product of <C> and <D> is not defined" );
    fi;
    return E;
endbrk> Parent(b.group);
G
brk> b in Parent(a.group);
true
brk> RightCoset( a.group, a.representative * b);
(Subgroup( A, [ Aut(G, [ s*d1, d1, d2 ]) ] )*InnerAut(G, d1))
brk>
brk> Print(b.operations.\*);
function ( a, b )
    local  prd;
    if IsAut( a ) and IsAut( b )  then
        if AutOps.IsKnownInner( a ) and AutOps.IsKnownInner( b )  then
            prd := AutOps.MakeAut( a.group, a.inner * b.inner );
        else
            prd := AutOps.MakeAut( a.group, AutOps.Compose( a, b ) );
            if IsBound( a.inverse ) and IsBound( b.inverse )  then
                prd.inverse
                 := AutOps.MakeAut( a.group, AutOps.Compose( b.inverse,
                     a.inverse ) );
                prd.inverse.inverse := prd;
            fi;
        fi;
    elif IsList( a )  then
        prd := List( a, function ( x )
                return x * b;
            end );
    elif IsList( b )  then
        prd := List( b, function ( x )
                return a * x;
            end );
    else
        Error( "product of <a> and <b> is not defined" );
    fi;
    return prd;
end
----------------------------------------------------------------------

I guess I have 2 questions:

(1) When asked to evaluate "a * b", why wasn't the "a.operations.\*"
function used first?

(2) How should I amend my "b.operations.\*" to cope with this situation?
Should I replace the last "else-error" clause with:

elif IsBound(a.operations.\*) then
return a.operations.\*(a,b);
else
Error( "product of <a> and <b> is not defined" );
fi;

This solution appears to work, and I'm happy to update my code, but it
would be useful to know the "correct" solution to this problem.

Cheers,
Michael.

-----------------------------------------------------------------------------
Michael.Smith@maths.anu.edu.au          DSTO & Australian National University