davrot/nnmf_24a
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update README.md

Browse source
This commit is contained in:
David Rotermund 2024-07-10 16:16:41 +02:00 committed by GitHub
parent 388b83c264
commit d8fc9f29a4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 69 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

69
README.md
View file

@ -1,3 +1,7 @@
Code excerpt from David Rotermund, Mahbod Nouri, Alberto Garcia-Ortiz and Kaus R. Pawelzik trying to understand deep NNMF networks.
# Origin of the algorithm
Refinement of the approach for deep NNMF networks shown in:
```
@ -16,3 +20,68 @@ David Rotermund and Klaus R. Pawelzik
https://www.frontiersin.org/journals/computational-neuroscience/articles/10.3389/fncom.2019.00055/full
```
# Network structure
Note: A block like
```
(1): Unfold(kernel_size=(5, 5), dilation=(1, 1), padding=(0, 0), stride=(1, 1))
(2): Fold(output_size=torch.Size([24, 24]), kernel_size=(1, 1), dilation=1, padding=0, stride=1)
(3): L1NormLayer()
(4): NNMF2d(75, 32, pfunctype=0, local_learning=False)
```
represents one(!) Con2d NNMF Layer. We just see more of the innards that for a normal Conv2d.
```
Sequential(
(0): ReLU()
(1): Unfold(kernel_size=(5, 5), dilation=(1, 1), padding=(0, 0), stride=(1, 1))
(2): Fold(output_size=torch.Size([24, 24]), kernel_size=(1, 1), dilation=1, padding=0, stride=1)
(3): L1NormLayer()
(4): NNMF2d(75, 32, pfunctype=0, local_learning=False)
(5): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=False)
(6): Conv2d(32, 32, kernel_size=(1, 1), stride=(1, 1))
(7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=False)
(8): ReLU()
(9): Unfold(kernel_size=(2, 2), dilation=(1, 1), padding=(0, 0), stride=(2, 2))
(10): Fold(output_size=torch.Size([12, 12]), kernel_size=(1, 1), dilation=1, padding=0, stride=1)
(11): L1NormLayer()
(12): NNMF2d(128, 32, pfunctype=0, local_learning=False)
(13): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=False)
(14): ReLU()
(15): Unfold(kernel_size=(5, 5), dilation=(1, 1), padding=(0, 0), stride=(1, 1))
(16): Fold(output_size=torch.Size([8, 8]), kernel_size=(1, 1), dilation=1, padding=0, stride=1)
(17): L1NormLayer()
(18): NNMF2d(800, 64, pfunctype=0, local_learning=False)
(19): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=False)
(20): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1))
(21): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=False)
(22): ReLU()
(23): Unfold(kernel_size=(2, 2), dilation=(1, 1), padding=(0, 0), stride=(2, 2))
(24): Fold(output_size=torch.Size([4, 4]), kernel_size=(1, 1), dilation=1, padding=0, stride=1)
(25): L1NormLayer()
(26): NNMF2d(256, 64, pfunctype=0, local_learning=False)
(27): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=False)
(28): ReLU()
(29): Unfold(kernel_size=(4, 4), dilation=(1, 1), padding=(0, 0), stride=(1, 1))
(30): Fold(output_size=torch.Size([1, 1]), kernel_size=(1, 1), dilation=1, padding=0, stride=1)
(31): L1NormLayer()
(32): NNMF2d(1024, 96, pfunctype=0, local_learning=False)
(33): Conv2d(96, 96, kernel_size=(1, 1), stride=(1, 1))
(34): ReLU()
(35): Unfold(kernel_size=(1, 1), dilation=(1, 1), padding=(0, 0), stride=(1, 1))
(36): Fold(output_size=torch.Size([1, 1]), kernel_size=(1, 1), dilation=1, padding=0, stride=1)
(37): L1NormLayer()
(38): NNMF2d(96, 10, pfunctype=0, local_learning=False)
(39): Conv2d(10, 10, kernel_size=(1, 1), stride=(1, 1))
(40): Softmax(dim=1)
(41): Flatten(start_dim=1, end_dim=-1)
)
Information about used parameters:
cnn_top: 14638
nnmf: 173344
batchnorm2d: 576
total number of parameter: 188558
```