Semiformal style question?

That's a fine casual look, but it's not semi-formal. Semi formal is just one step below formal (as the name implies), so men have to wear dress pants (khakis aren't dressy enough), shirts, ties, dress shoes, and preferably jackets. You might be able to get away with no jacket depending on the nature of the event, but you definitely have to wear formal pants and shoes.