It depends on the images, if you needed a lot of details from the image you don't need to increase your kernel size. ... Smaller kernel will gives you a lot of details, it can lead you to overfitting, but larger kernel will gives you loss a lot of details, it can lead you to underfitting