"I have panic attacks and social anxiety so would want to use the CBD when I have a panic attack to calm the panic and excitablility which is horrible."
In terms of weaning any medication, such should be done under the guidance of your primary care provider.
If you are using lorazepam on an as needed basis and have limited requirements it is safe to say that you can likely wean yourself as the medication is intended for symptoms as required. Having said that, cannabis should not be used in conjunction with any sedating medication like lorazepam due to the risk of oversedation.
Hope that helps!