How can I create multiple web items at once? 

On your standard item list, set your filters and select the items for which you want to create web items. Then under Process click Create Magento Item(s).