Machine-Learning
/
AIIA
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

gates where not saved as well #25

Merged
Fabel merged 1 commits from bug_fixes into main 2025-03-03 18:02:01 +00:00
Conversation 0 Commits 1 Files Changed 1 +2 -2
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
src/aiia/model/Model.py
View File

@ -193,8 +193,8 @@ class AIIAmoe(AIIA):
# To generate gating weights, we first need to determine the feature dimension.
# Each expert is assumed to return an output of shape (B, C, H, W); after averaging over H and W,
# we obtain a tensor of shape (B, C) where C is the number of channels (here assumed to be 410).
gate_in_features = 410 # Adjust this if your expert output changes.
# we obtain a tensor of shape (B, C) where C is the number of channels (here assumed to be 224).
gate_in_features = 512 # Adjust this if your expert output changes.
# Create a gating network that maps the aggregated features to num_experts weights.
self.gate = nn.Sequential(