Usage of the word "refuse"

I have a question regarding the proper usage of the word "refuse"

Let's say someone made a request to me, and my desire is to grant the request to the person. However let's assume that I am unable to grant the request despite my desire because the request entails something that is impossible for me to do. So if I say to this person, "sorry, I can't grant your request although I wish that I can do it for you", am I "refusing" to grant the request? or is there a better way to put it?