To add a lesson to the Lessons library initially you need to visit the course's dashboard (1). Over there clicking on the Add content button (2) a dropdown menu will open with various options.
Click on the Go to Lessons Library (3) option to view all lessons in the system along with those assigned to that course.
To add a new lesson click on the Add lesson (4) button. Clicking on that option will open a pop up form with the details lesson's details that you need to provide. You need to fill the lesson's name and an optional description of the lesson.
Once you complete filling in the new lesson form click Add to add the new lesson. You will get redirected back to the course's dashboard and the new lesson you added will automatically get assigned to the course you are currently browsing.
Note: When you add a new course and create its first unit, a new lesson is automatically created and is available in the Lessons library. It won't show up in the course dashboard though, unless a second lesson is added to the course.