How does nn.conv2D implement the feature map table of LeNet-5? #16
Comments
|
Hi Tom, The PyTorch implementation of C3 layer is a "reinterpretation" if you will of the original paper. It does not reflect the fancy customized column assignment of the depicted Table 1. I guess LeCun did it like that back then to try to squeeze some performance out of it - not something we need to be concerned anymore. Regarding the IPython imports, I've only used them in the very first cell of the notebook to make it wider - if you're running it locally, it will make visualization a bit nicer, but that's all there is to it. Best, |
|
On Sun, Dec 27, 2020 at 5:45 AM Daniel Voigt Godoy ***@***.***> wrote:
Hi Tom,
The PyTorch implementation of C3 layer is a "reinterpretation" if you will
of the original paper. It does not reflect the fancy customized column
assignment of the depicted Table 1. I guess LeCun did it like that back
then to try to squeeze some performance out of it - not something we need
to be concerned anymore.
Regarding the IPython imports, I've only used them in the very first cell
of the notebook to make it wider - if you're running it locally, it will
make visualization a bit nicer, but that's all there is to it.
Best,
Daniel
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#16 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADHGGHBTN6AALEHB3PE7WLDSW426NANCNFSM4VITCUVQ>
.
Got it!
Thank You
Tom
|
|
Hi Tom, Just published a new chapter :-) Best, |
|
Daniel,
No, I did not know about the release of chapter 16 until you told me. Of
course, I could have been checking now and then. My bad.
In the meantime, I started another tutorial with "Hands on machine learning
with Scikit learn, Keras and Tensorflow". Let me tell you that I wish YOU
were the one that had written it. Your writing style is much clearer.
Thank You
Tom
…On Thu, Jan 14, 2021 at 1:10 PM Daniel Voigt Godoy ***@***.***> wrote:
Hi Tom,
Just published a new chapter :-)
Not sure if you're receiving notifications from Leanpub, so I thought of
letting you know here.
Have fun!
Best,
Daniel
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#16 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADHGGHFTRSRD4S4V5FZO3UDSZ5MVLANCNFSM4VITCUVQ>
.
|
|
Daniel,
I am sorry about the confusion. I got mixed up between your pytorch step by
step course and your Machine learning course fundamentals course. I think
that I did get the new chapter.
One question that I have is that I was going though parts of the machine
learning course and I could not find a way to load the linear algebra
module
*intuitiveml.algebra.LinearAlgebra. *Any idea of where I can find it?
Thank You
Tom
On Thu, Jan 14, 2021 at 7:30 PM Tom Cipollone <minertom11551@gmail.com>
wrote:
… Daniel,
No, I did not know about the release of chapter 16 until you told me. Of
course, I could have been checking now and then. My bad.
In the meantime, I started another tutorial with "Hands on machine
learning with Scikit learn, Keras and Tensorflow". Let me tell you that I
wish YOU were the one that had written it. Your writing style is much
clearer.
Thank You
Tom
On Thu, Jan 14, 2021 at 1:10 PM Daniel Voigt Godoy <
***@***.***> wrote:
> Hi Tom,
>
> Just published a new chapter :-)
> Not sure if you're receiving notifications from Leanpub, so I thought of
> letting you know here.
> Have fun!
>
> Best,
> Daniel
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#16 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADHGGHFTRSRD4S4V5FZO3UDSZ5MVLANCNFSM4VITCUVQ>
> .
>
|
|
Hi, Thanks for your comment on my writing style, that means a lot to me :-) Regarding the LinearAlgebra module: I've developed this package as base for the ML Fundamentals course, but it was kinda "hacky" so I've uploaded it to PyPI only (but not GitHub). If you install it using Also, the code for "intuitiveml" is open, you can download its source from PyPI here: https://pypi.org/project/intuitiveml/#files Hope it helps. Best, |
|
Hi Danial,
I know that you are based in Germany, or the Netherlands, so, the chances
of my going to anything but an online class taught by you are probably low.
But, do you have a video course online or one planned for the future?
i would really like to see your "take" on support vector machines. As I
mentioned earlier, I am going through "hands on machine learning" by Geron.
Definitely not a good introduction to some of the topics. Finding that I
had to go far afield from the actual book in order to find good material
for making the topic "real".
Regards
Tom Cipollone
…On Fri, Jan 15, 2021 at 4:23 AM Daniel Voigt Godoy ***@***.***> wrote:
Hi,
Thanks for your comment on my writing style, that means a lot to me :-)
Regarding the LinearAlgebra module: I've developed this package as base
for the ML Fundamentals course, but it was kinda "hacky" so I've uploaded
it to PyPI only (but not GitHub).
If you install it using pip install intuitiveml, you should be able to
import the LinearAlgebra module and use it as you like.
Also, the code for "intuitiveml" is open, you can download its source from
PyPI here: https://pypi.org/project/intuitiveml/#files
Hope it helps.
Best,
Daniel
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#16 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADHGGHBTJUZTVMK76CMVSG3S2AXTNANCNFSM4VITCUVQ>
.
|
|
Hi Tom, I don't have a video course yet, but it is definitely in my plans for 2021 after I finish the book. Regarding SVMs, I guess you've already gone through the corresponding notebook in ML Fundamentals (https://github.com/dvgodoy/ML_Fundamentals/blob/master/13.%20Support%20Vector%20Machine%20(SVM).ipynb)? There isn't much to it actually: it is a glorified linear regression. The trick is, you want to have all points in the positive class to have the "predicted y" being 1 or more, while all points in the negative class must have their "predicted y" being -1 or less. The +1/-1 is completely arbitrary. Moreover, you want two points, the extremes of each class, to be exactly 1 and -1. There is one set of weights for the "y = w1x1 + w2x2 +..." that makes that happen (IF the points are separable by a straight line). Since most real problems won't be separable like that, you need to find a compromise between this linear regression and the points that are trespassing. That's a soft margin, and you control how soft/hard it is by using the C multiplier on the penalty of the trespasser points. Then there is the kernel trick, which is just a fancy way of applying a function to all the features (usually polynomials) to increase dimensionality and make the points more easily separable. The kernel trick itself is just a mathematical shortcut to avoid actually computing these transformations. Roughly speaking, it is like: the function of the dot product yields the same result as the dot product of the function. Why dot product, right? Again, fancy mathematical thing... minimizing the weights of that linear regression is equivalent to a different optimization problem that involves computing dot products for all pairs of data points. SVMs are a clever way of addressing nonlinearities in datasets, especially before NNs and activation functions became the norm. For some reason, it is still common to see them used in medicine/healthcare, but not so much anywhere else (that I know of). I am not sure if that's what you were looking for... is there any other specifics you were interested in? Best, |
|
Daniel,
Thank you so much for your kind input. Now that I know a little more about
SVM I don't believe that it is worth much in the line of research that I am
pursuing. I had expected that I would find a good mathematical explanation
of the SVM as compared to the Gilbert Strang from MIT Linear algebra
lectures #15 and #16 which described Linear Regression and the Normal
equations very clearly. No such luck with SVM. In any case, CNNs seem to be
"the thing" best suited for object detection.
My interest is in object detection. You may remember that I already used
YOLOV3 in a project of object detection of aircraft using the darknet
framework. I'll never use the darknet framework again because it is not
very flexible and the community that uses it is very small.
There is a version of YOLOV5 using pytorch that was put on github by
ultralytics https://github.com/ultralytics/yolov5.
<https://github.com/ultralytics/yolov5>
There is also a version of YOLOV3 using tensorflow 2 from
https://github.com/zzh8829/yolov3-tf2.
At this point, I feel somewhat just a little competent about the Pytorch
framework, thanks to your book, but I want to be able to compare it to the
tensorflow framework.
Please let me know when you are working on the course.
Thank You
Tom
…On Wed, Jan 20, 2021 at 11:36 AM Daniel Voigt Godoy < ***@***.***> wrote:
Hi Tom,
I don't have a video course *yet*, but it is definitely in my plans for
2021 after I finish the book.
Regarding SVMs, I guess you've already gone through the corresponding
notebook in ML Fundamentals (
https://github.com/dvgodoy/ML_Fundamentals/blob/master/13.%20Support%20Vector%20Machine%20(SVM).ipynb
)?
There isn't much to it actually: it is a glorified linear regression. The
trick is, you want to have all points in the positive class to have the
"predicted y" being 1 or more, while all points in the negative class must
have their "predicted y" being -1 or less. The +1/-1 is completely
arbitrary. Moreover, you want two points, the extremes of each class, to be
*exactly* 1 and -1. There is one set of weights for the "y = w1x1 + w2x2
+..." that makes that happen (IF the points are separable by a straight
line).
Since most real problems *won't* be separable like that, you need to find
a compromise between this linear regression and the points that are
*trespassing*. That's a soft margin, and you control how soft/hard it is
by using the C multiplier on the penalty of the trespasser points.
Then there is the kernel trick, which is just a fancy way of applying a
function to all the features (usually polynomials) to increase
dimensionality and make the points more easily separable. The kernel trick
itself is just a mathematical shortcut to avoid actually computing these
transformations. Roughly speaking, it is like: the function of the dot
product yields the same result as the dot product of the function.
Why dot product, right? Again, fancy mathematical thing... minimizing the
weights of that linear regression is equivalent to a *different*
optimization problem that involves computing dot products for all pairs of
data points.
SVMs are a clever way of addressing nonlinearities in datasets, especially
before NNs and activation functions became the norm. For some reason, it is
still common to see them used in medicine/healthcare, but not so much
anywhere else (that I know of).
I am not sure if that's what you were looking for... is there any other
specifics you were interested in?
Best,
Daniel
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#16 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADHGGHHUAICRYT4FYB2WXULS24WDHANCNFSM4VITCUVQ>
.
|
|
Hi Daniel,
After having worked on "Hands on machine Learning with Scikit Learn, Keras
and Tensorflow" for a couple of days, I have decided to put it aside. I
find it far to obscure for the equivalent functionality of pytorch.
Therefore, I will be reviewing your course/book again and concentrating on
object detection with pytorch. For my use I am finding Keras/Tensorflow not
worth the effort.
Regards
Tom
On Wed, Jan 20, 2021 at 4:45 PM Tom Cipollone <minertom11551@gmail.com>
wrote:
… Daniel,
Thank you so much for your kind input. Now that I know a little more about
SVM I don't believe that it is worth much in the line of research that I am
pursuing. I had expected that I would find a good mathematical explanation
of the SVM as compared to the Gilbert Strang from MIT Linear algebra
lectures #15 and #16 which described Linear Regression and the Normal
equations very clearly. No such luck with SVM. In any case, CNNs seem to be
"the thing" best suited for object detection.
My interest is in object detection. You may remember that I already used
YOLOV3 in a project of object detection of aircraft using the darknet
framework. I'll never use the darknet framework again because it is not
very flexible and the community that uses it is very small.
There is a version of YOLOV5 using pytorch that was put on github by
ultralytics https://github.com/ultralytics/yolov5.
<https://github.com/ultralytics/yolov5>
There is also a version of YOLOV3 using tensorflow 2 from
https://github.com/zzh8829/yolov3-tf2.
At this point, I feel somewhat just a little competent about the Pytorch
framework, thanks to your book, but I want to be able to compare it to the
tensorflow framework.
Please let me know when you are working on the course.
Thank You
Tom
On Wed, Jan 20, 2021 at 11:36 AM Daniel Voigt Godoy <
***@***.***> wrote:
> Hi Tom,
>
> I don't have a video course *yet*, but it is definitely in my plans for
> 2021 after I finish the book.
>
> Regarding SVMs, I guess you've already gone through the corresponding
> notebook in ML Fundamentals (
> https://github.com/dvgodoy/ML_Fundamentals/blob/master/13.%20Support%20Vector%20Machine%20(SVM).ipynb
> )?
>
> There isn't much to it actually: it is a glorified linear regression. The
> trick is, you want to have all points in the positive class to have the
> "predicted y" being 1 or more, while all points in the negative class must
> have their "predicted y" being -1 or less. The +1/-1 is completely
> arbitrary. Moreover, you want two points, the extremes of each class, to be
> *exactly* 1 and -1. There is one set of weights for the "y = w1x1 + w2x2
> +..." that makes that happen (IF the points are separable by a straight
> line).
>
> Since most real problems *won't* be separable like that, you need to
> find a compromise between this linear regression and the points that are
> *trespassing*. That's a soft margin, and you control how soft/hard it is
> by using the C multiplier on the penalty of the trespasser points.
>
> Then there is the kernel trick, which is just a fancy way of applying a
> function to all the features (usually polynomials) to increase
> dimensionality and make the points more easily separable. The kernel trick
> itself is just a mathematical shortcut to avoid actually computing these
> transformations. Roughly speaking, it is like: the function of the dot
> product yields the same result as the dot product of the function.
>
> Why dot product, right? Again, fancy mathematical thing... minimizing the
> weights of that linear regression is equivalent to a *different*
> optimization problem that involves computing dot products for all pairs of
> data points.
>
> SVMs are a clever way of addressing nonlinearities in datasets,
> especially before NNs and activation functions became the norm. For some
> reason, it is still common to see them used in medicine/healthcare, but not
> so much anywhere else (that I know of).
>
> I am not sure if that's what you were looking for... is there any other
> specifics you were interested in?
>
> Best,
> Daniel
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#16 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADHGGHHUAICRYT4FYB2WXULS24WDHANCNFSM4VITCUVQ>
> .
>
|
I have been going through the pytorch documentation of conv2D. Reading the docs from pytorch I see
torch.nn.Conv2d(in_channels: int, out_channels: int, kernel_size: Union[T, Tuple[T, T]], stride: Union[T, Tuple[T, T]] = 1, padding: Union[T, Tuple[T, T]] = 0, dilation: Union[T, Tuple[T, T]] = 1, groups: int = 1, bias: bool = True, padding_mode: str = 'zeros')From the LeNet paper of November 1998 I see that the third convolution layer is implemented with 6 input layers and 16 output layers. The 16 output layers are made from a combination of the 6 input layers according to a table, also in the paper:
From the chapter 5 tutorial, C3 is implemented with
lenet.add_module('C3', nn.Conv2d(in_channels=6, out_channels=16, kernel_size=5))What I do not see is how pytorch implements this feature map table. It seems a little like a "loaves and fishes" issue :-).
P.S. If I might kill two birds with one stone, can i ask which plots are being implemented with IPython? I have observed that usually the plots that you generate are implemented in the plots directories from each chapter, using matplotlib. I was looking for HTML plots that were generated with IPython, because you call the IPython.core.display module and import display, HTML. But, I could not seem to spot how it was used.
Thank You
Tom
The text was updated successfully, but these errors were encountered: