Error per context?

[lkmartin90]

Hi,
Thanks for the great package. I just wondered if there's an easy way to return the error in the factorisation per context. If my understanding of the method is correct, there's a chance that the factors could fit (N-1)/N contexts very well while describing the Nth context very badly, but still be the best fit for the tensor as a whole. I just wanted to check that this wasn't the case.

[earmingol]

This is a great and interesting question indeed! It is possible that some contexts have more error than others, but we do not evaluate that.

What I could think of an easy way of doing it by context is using your original tensor, subset by the context you are interested in and then do the same for the reconstructed tensor, subset it by the same context and compute the error. A quick pseudocode would be:

context_tensor = interaction_tensor.tensor[1, :, :, :] # where 1 could be another number depending on your context index
context_rec_tensor = interaction_tensor.tl_object.cp_to_tensor() [1, :, :, :] # where 1 could be another number depending on your context index
err = cell2cell.tensor.factorization.normalized_error(context_tensor, context_rec_tensor)

Let me know if this works or you have any questions

[lkmartin90]

Thanks for such a quick reply! I changed it slightly to get it to work for me, although I was expecting an error between 0 and 1 and actually get 2.something - but I'll read the c2c.tensor.factorization code to get my head around that.

import tensorly as tl

context_tensor = tensor_final.tensor[1, :, :, :] # Adjust index as needed

context_rec_tensor = tl.cp_to_tensor(tensor_final.tl_object)[1, :, :, :] # Adjust index as needed

err = c2c.tensor.factorization.normalized_error(context_tensor, context_rec_tensor)

Edit: stupid mistake, the last line should be err = c2c.tensor.factorization.normalized_error(context_rec_tensor, context_tensor) for sensible values

[lkmartin90]

Sorry to add to this further - the method gives much higher error than expected from the elbow plot output, even when the full tensor is used rather than a slice. My understanding of the tensor package isn't good enough to work out what is going wrong.
Thanks again!

[earmingol]

So sorry for the very delayed answer. I thought I already did it but just realized I did not!

In case this helps, the issue of having weird behaviors with the errors could be due to high sparsity of your tensor. Probably you need to be more strict and instead of using cells or LR that are context-specific only keep those that are shared across a high percentage of samples/context (outer_fraction=0.8 when using how='outer'). You can always check your sparsity or how many NaN values you have with tensor_final.sparsity_fraction() and tensor_final.missing_fraction()

I hope this helps!