Grow Deeper to Reach Wider