Forcing a Signature

A while ago I added a bool to the Signature class that allowed the user to specify that the signature was already fully formed and did not need any processing. This was part of the "symbol file" hack that we used to do procedure-at-a-time decompilation using the command line tool. I noticed today that we were not honouring the forced bit anymore, for example, we were removing unused parameters and returns from the signature, so I fixed that. It occured to me that any procedure we discover via a function pointer was an excellent candidate for setting the forced bit on. The results were pretty spectacular as locals and globals were soon inheriting type information from the parameters of the signature. Unfortunately, the same could not be said of returns. In particular, consider some code like this:

mystruct *proc5()
12 { *v** r24 } := malloc(343)
13 m[r24{12} + 4] := "foo"
14 RET r24{12}

It's pretty clear that any local we create for r24 should be of type mystruct* but the first definition of r24 has us assigning it a type of void*. We could use some kind of lattice logic to determine that mystruct* is "better" than void* but the fact that the signature is forced tells us that this is no debate, r24 must have the type mystruct*. This analysis can be done in the early decompilation stage, but where to put the type? At the moment I'm considering changing the type of the return in the call statement. So line 12 will become:

12 {*mystruct** r24 } := malloc(343)

and immediately this type will flow on to turn statement 13 into something similar to:

13 m[r24{12}].name := "foo"

which is much more preferable.