ANSWER: Cain's sins separated him from God (Isaiah 59:2). His attitude and actions were not right with God. In Genesis 4:3-7 we read: "And in the process of time it came to pass that Cain brought an offering of the fruit of the ground to the LORD. Abel also brought of the firstlings of his flock and of their fat. And the LORD respected Abel and his offering, but He did not respect Cain and his offering. And Cain was very angry, and his countenance fell. So the LORD said to Cain, 'Why are you angry? And why has your countenance fallen? If you do well, will you not be accepted? And if you do not do well, sin lies at the door. And its desire is for you, but you should rule over it." Because Cain was in a rebellious state of mind, God would not have accepted his sacrifice even if he had offered an animal. Proverbs 15:8 states: "The sacrifice of the wicked is an abomination to the LORD, but the prayer of the upright is His delight."
Why was Abel's offering accepted and Cain's was not? Abel was a righteous man of faith (Hebrews 11:4). Cain, on the other hand, rejected God and His laws and lived an evil life. "Not as Cain who was of the wicked one and murdered his brother. And why did he murder him? Because his works were evil and his brother's righteous" (I John 3:12). God admonished Cain to overcome the sin that permeated his life. But Cain did not follow God's instruction, but chose to reject God and His laws, and eventually even murdered his own brother (Genesis 4:8).