I have large inbox; does this mean the caching process will take longer?

No, the caching process is dependent upon the number and size of new mail messages you receive.