No you can upload one by one.
An Amazon listing can contain up to six videos. Both the Amazon Video Shorts section and videos added to the product image block fall under this category. Your chances of showing up in search results might be increased by adding more than one video.
