Model-Free Non-Stationary Reinforcement Learning: Near-Optimal Regret and Applications in Multi-Agent Reinforcement Learning and Inventory Control