If you are online (dialup) and using your phone line then it is obvious that the fax can’t dial out and reports the line is in use..
Modems usually have a phone in and a phone out jack. fax machines usually only have 1 phone jack (phone out). You could try a 2-1 phone jack splitter (available at most dollar stores (3 or 4-1 are more common and serve same purpose)
A fax is a special type of modem almost the same as what you are using to go online..
Either way you can either talk on the phone, go online, or send a fax at one time.. you can not do two things at once.. also when you are online people can’t call/fax you.