QBoard » Artificial Intelligence & ML » AI and ML - Conceptual » Convolutional Deep Belief Networks (CDBN) vs. Convolutional Neural Networks (CNN)
User Dashboard

Convolutional Deep Belief Networks (CDBN) vs. Convolutional Neural Networks (CNN)

Back To Topics
Tags : None
  • 126
    Lastly, I started to learn neural networks and I would like know the difference between Convolutional Deep Belief Networks and Convolutional Networks. In here, there is a similar question but there is no exact answer for it. We know that Convolutional Deep Belief Networks are CNNs + DBNs. So, I am going to do an object recognition. I want to know which one is much better than other or their complexity. I searched but I couldn't find anything maybe doing something wrong.
      November 22, 2021 6:55 PM IST
    0
  • 346 3

    Deep Learning is the branch of Machine Learning based on Deep Neural Networks (DNNs), meaning neural networks with at the very least 3 or 4 layers (including the input and output layers). But for some people (especially non-technical), any neural net qualifies as Deep Learning, regardless of its depth. And others consider a 10-layer neural net as shallow.

    Convolutional Neural Networks (CNNs) are one of the most popular neural network architectures. They are extremely successful at image processing, but also for many other tasks (such as speech recognition, natural language processing, and more). The state of the art CNNs are pretty deep (dozens of layers at least), so they are part of Deep Learning. But you can build a shallow CNN for a simple task, in which case it's not (really) Deep Learning.

    But CNNs are not alone, there are many other neural network architectures out there, including Recurrent Neural Networks (RNN), Autoencoders, Transformers, Deep Belief Nets (DBN = a stack of Restricted Boltzmann Machines, RBM), and more. They can be shallow or deep. Note: even shallow RNNs can be considered part of Deep Learning since training them requires unrolling them through time, resulting in a deep net.

      December 17, 2021 11:49 AM IST
    0
  • 126
    I don't know if you still need an answer but anyway I hope you will find this useful.
    A CDBN adds the complexity of a DBN, but if you already have some background it's not that much.
    If you are worried about computational complexity instead, it really depends on how you use the DBN part. The role of DBN usually is to initialize the weights of the network for faster convergence. In this scenario, the DBN appears only during pre-training.
    You can also use the whole DBN like a discriminative network (keeping the generative power) but the weight initialization provided by it is enough for discriminative tasks. So during an hypothetical real-time utilization, the two system are equal performance-wise.
    Also the weight-initialization provided by the first model anyway really helps for difficult task like object recognition (even a good Convolutional Neural network alone doesn't reach good success rate, at least compared to a human) so it's generally a good choice.
      November 23, 2021 5:37 PM IST
    0
  • 319 1

    Generally speaking, DBNs are generative neural networks that stack Restricted Boltzmann Machines (RBMs) . You can think of RBMs as being generative autoencoders; if you want a deep belief net you should be stacking RBMs and not plain autoencoders as Hinton and his student Yeh proved that stacking RBMs results in sigmoid belief nets.

    Convolutional neural networks have performed better than DBNs by themselves in current literature on benchmark computer vision datasets such as MNIST. If the dataset is not a computer vision one, then DBNs can most definitely perform better. In theory, DBNs should be the best models but it is very hard to estimate joint probabilities accurately at the moment. You may be interested in Lee et. al's (2009) work on Convolutional Deep Belief Networks which looks to combine the two.

      November 24, 2021 11:50 AM IST
    0