Bringing legal knowledge to the public by constructing a legal question bank using large-scale pre-trained language model

Abstract

Access to legal information is fundamental to access to justice. Yet accessibility refers not only to making legal documents available to the public, but also rendering legal information comprehensible to them. A vexing problem in bringing legal information to the public is how to turn formal legal documents such as legislation and judgments, which are often highly technical, to easily navigable and comprehensible knowledge to those without legal education. In this study, we formulate a three-step approach for bringing legal knowledge to laypersons, tackling the issues of navigability and comprehensibility. First, we translate selected sections of the law into snippets (called CLIC-pages), each being a small piece of article that focuses on explaining certain technical legal concept in layperson’s terms. Second, we construct a Legal Question Bank, which is a collection of legal questions whose answers can be found in the CLIC-pages. Third, we design an interactive CLIC Recommender. Given a user’s verbal description of a legal situation that requires a legal solution, CRec interprets the user’s input and shortlists questions from the question bank that are most likely relevant to the given legal situation and recommends their corresponding CLIC pages where relevant legal knowledge can be found. In this paper we focus on the technical aspects of creating an LQB. We show how large-scale pre-trained language models, such as GPT-3, can be used to generate legal questions. We compare machine-generated questions against human-composed questions and find that MGQs are more scalable, cost-effective, and more diversified, while HCQs are more precise. We also show a prototype of CRec and illustrate through an example how our 3-step approach effectively brings relevant legal knowledge to the public.

Appendix A Questions created by human workers versus those composed by Machine

We show the questions generated by human workers and those by various machine methods for CLIC-page $p_3$: “Repairing Maintenance Obligations” (https://www.clic.org.hk/en/topics/landlord_tenant/repairing_MaintenanceObligations/q1). The answer scope of a question is shown in parentheses at the end of the question. For questions generated by Machine that are incorrect, their answer scopes are marked “N.A.” (for “No Answer”).

[Questions composed by human workers]:

• - [Q01] “In general, how do parties specify maintenance responsibility in tenancy agreements?” ($p_3$:[heading])

• - [Q02] “Who should be responsible for external and structural maintenance?” ($p_3$:[2])

• - [Q03] “Who should be responsible for internal and non-structural maintenance?” ($p_3$:[2])

• - [Q04] “What kind of maintenance should be responsible by the landlord?” ($p_3$:[2])

• - [Q05] “What kind of maintenance should be responsible by the tenant?” ($p_3$:[2])

• - [Q06] “Are the terms like external, internal, structural, or non-structural clear enough to specify maintenance responsibility?” ($p_3$:[2, 3])

• - [Q07] “Is it fair for a tenant to become responsible for property maintenance?” ($p_3$:[4])

• - [Q08] “Do tenants have the responsibility to maintain the property from "fair wear and tear"?” ($p_3$:[5])

• - [Q09] “Without the notice of structural defects, is the landlord responsible for structural repair?” ($p_3$:[5])

• - [Q10] “Is it reasonable for the landlord to be notified before being responsible for structural repair?” ($p_3$:[5])

• - [Q11] “Why some landlord may volunteer to carry out repairs and maintenance works?” ($p_3$:[7])

• - [Q12] “What are the advantages for landlords to repair and maintain the property voluntarily?” ($p_3$:[7])

• - [Q13] “If tenants become incooperative to landlord carrying out necessary inspection and repair works, what can the landlord do?” ($p_3$:[8])

[Questions generated by Hybrid]:

• - [Q01] “What should I do if I have a problem with my property that needs to be repaired or maintained?” (N.A.)

• - [Q02] “Who is responsible for external and structural repairs and maintenance?” ($p_3$:[2])

• - [Q03] “Who is responsible for internal and non-structural repairs and maintenance?” ($p_3$:[2])

• - [Q04] “What is the meaning of "fair wear and tear excepted"?” ($p_3$:[5])

• - [Q05] “What is the landlord’s obligation for structural repairs and maintenance?” ($p_3$:[2])

• - [Q06] “How can I find out who is responsible for repairs and maintenance?” ($p_3$:[1])

• - [Q07] “Can the landlord volunteer to carry out repairs and maintenance works?” ($p_3$:[7])

• - [Q08] “What if the tenant becomes uncooperative?” ($p_3$:[8])

• - [Q09] “Can the landlord apply for an injunction?” ($p_3$:[8])

• - [Q10] “Can the landlord terminate the tenancy agreement?” ($p_3$:[8])

• - [Q11] “What is the difference between external and internal repairs?” ($p_3$:[2])

• - [Q12] “What is the difference between structural and non-structural repairs?” ($p_3$:[2])

• - [Q13] “Who is responsible for repair and maintenance of the property?” ($p_3$:[heading])

• - [Q14] “What is the difference between "fair wear and tear" and "damage"?” ($p_3$:[5])

• - [Q15] “What is the landlord’s obligation for structural repairs and maintenance?” ($p_3$:[2])

• - [Q16] “What is the tenant’s obligation for repair and maintenance of the property?” ($p_3$:[heading])

• - [Q17] “Can the landlord carry out repair and maintenance works without the tenant’s consent?” ($p_3$:[8])

• - [Q18] “What if the tenant refuses to allow the landlord to carry out repair and maintenance works?” ($p_3$:[8])

• - [Q19] “Can the landlord terminate the tenancy agreement if the tenant refuses to allow him/her to carry out repair and maintenance works?” ($p_3$:[8])

• - [Q20] “What can the tenant do if he/she is not satisfied with the repair and maintenance works carried out by the landlord?” (N.A.)

• - [Q21] “What is a typical approach under tenancy agreements when it comes to repair and maintenance responsibilities?” ($p_3$:[2])

• - [Q22] “How can a simple dichotomy between landlord and tenant responsibilities for repair and maintenance be problematic?” ($p_3$:[2])

• - [Q23] “What should a well-drafted tenancy document do in order to clarify the parties’ duties with regards to repair and maintenance?” ($p_3$:[3])

• - [Q24] “Is it common for tenants to be responsible for onerous duties such as repair and maintenance under their tenancy agreement?” ($p_3$:[4])

• - [Q25] “What is the significance of the phrase "fair wear and tear excepted" with regards to a tenant’s obligations for repair and maintenance?” ($p_3$:[5])

• - [Q26] “What is the role of notice clauses in specifying a landlord’s obligations for structural repairs and maintenance?” ($p_3$:[5])

• - [Q27] “Why might landlords agree to incur expenses to remedy defects in the property even though they are not contractually obliged to do so?” ($p_3$:[7])

• - [Q28] “In what circumstances might a landlord exercise a right to enter the property and carry out necessary inspection and repair works?” ($p_3$:[8])

• - [Q29] “What could happen if a tenant becomes uncooperative when a landlord tries to exercise such a right?” ($p_3$:[8])

• - [Q30] “Is it possible for a landlord to terminate a tenancy agreement if a tenant does not cooperate with repair and maintenance works?” ($p_3$:[8])

• - [Q31] “What is the fair wear and tear exception?” ($p_3$:[5])

• - [Q32] “What does it mean when a tenancy agreement states that the tenant’s obligations for repair and maintenance are limited by the phrase "fair wear and tear excepted"?” ($p_3$:[5])

• - [Q33] “What is the landlord’s obligation for structural repairs and maintenance?” ($p_3$:[2])

• - [Q34] “What is the tenant’s obligation for repair and maintenance?” ($p_3$:[heading])

• - [Q35] “How can I find out who is responsible for repairs and maintenance?” ($p_3$:[1])

• - [Q36] “How can I find out if the landlord is responsible for structural repairs and maintenance?” ($p_3$:[1, 2, 6])

• - [Q37] “How can I find out if the tenant is responsible for repair and maintenance?” ($p_3$:[1, 2, 6])

• - [Q38] “Is it common for a tenant to be responsible for repair and maintenance?” ($p_3$:[4])

• - [Q39] “What is the landlord’s right to enter the property and carry out necessary inspection and repair works?” ($p_3$:[8])

• - [Q40] “Can the landlord terminate the tenancy agreement if the tenant is uncooperative?” ($p_3$:[8])

• - [Q41] “What is the fair wear and tear exception?” ($p_3$:[5])

• - [Q42] “What does it mean when a tenant’s obligations for repair and maintenance are limited by the phrase "fair wear and tear excepted"?” ($p_3$:[5])

• - [Q43] “What is included in the fair wear and tear exception?” ($p_3$:[5])

• - [Q44] “How is the fair wear and tear exception determined?” (N.A.)

• - [Q45] “What are some examples of fair wear and tear?” (N.A.)

• - [Q46] “Is there a limit to the amount of damage that can be caused by fair wear and tear?” (N.A.)

• - [Q47] “What if the tenant causes damage that is not considered to be fair wear and tear?” (N.A.)

• - [Q48] “Can the landlord repair or replace items that are damaged by fair wear and tear?” ($p_3$:[5, 7])

• - [Q49] “Who is responsible for repairing or replacing items that are damaged by fair wear and tear?” ($p_3$:[5, 7])

• - [Q50] “What if the landlord and tenant cannot agree on what is considered to be fair wear and tear?” (N.A.)

• - [Q51] “What is the "fair wear and tear" exception in a tenancy agreement?” ($p_3$:[5])

• - [Q52] “What is the landlord’s obligation for structural repairs and maintenance?” ($p_3$:[2])

• - [Q53] “How can I tell if the damage is due to "fair wear and tear" or not?” ($p_3$:[5])

• - [Q54] “I am a tenant. Can I carry out repair and maintenance works on my own?” ($p_3$:[4])

• - [Q55] “I am a landlord. Can I enter the property to carry out repair and maintenance works without the tenant’s permission?” ($p_3$:[8])

• - [Q56] “My tenancy agreement does not contain any clauses regarding repair and maintenance. Who is responsible for repairs and maintenance?” ($p_3$:[heading])

• - [Q57] “My tenancy agreement contains a clause stating that the tenant is responsible for all repair and maintenance works. Is this fair?” ($p_3$:[1, 4])

• - [Q58] “My tenancy agreement contains a clause stating that the landlord is responsible for all repair and maintenance works. Is this fair?” ($p_3$:[1, 7])

• - [Q59] “I am a tenant. The property I am renting is in a state of disrepair. What can I do?” ($p_3$:[heading])

• - [Q60] “I am a landlord. The property I am renting out is in a state of disrepair. What can I do?” ($p_3$:[heading])

• - [Q61] “Can the landlord enter the property without the tenant’s permission?” ($p_3$:[8])

• - [Q62] “What if the tenant refuses to allow the landlord to enter the property?” ($p_3$:[8])

• - [Q63] “What if the tenant is not cooperative?” ($p_3$:[8])

• - [Q64] “What if the landlord needs to carry out urgent repairs?” ($p_3$:[8])

• - [Q65] “What if the tenancy agreement does not impose any duty on the landlord?” ($p_3$:[6, 7, 8])

• - [Q66] “Can the landlord terminate the tenancy agreement?” ($p_3$:[8])

• - [Q67] “What if the tenant objects to the repairs?” ($p_3$:[8])

• - [Q68] “Can the landlord charge the tenant for the cost of repairs?” (N.A.)

• - [Q69] “What if the repairs are not carried out properly?” (N.A.)

• - [Q70] “Can the tenant sue the landlord for breach of contract?” (N.A.)

• - [Q71] “What is the definition of "serious water leakage"?” (N.A.)

• - [Q72] “What is the landlord’s right to enter the property?” ($p_3$:[8])

• - [Q73] “What is the landlord’s duty to carry out necessary inspection and repair works?” ($p_3$:[8])

• - [Q74] “What is the tenant’s obligation to cooperate with the landlord?” ($p_3$:[8])

• - [Q75] “What are the consequences if the tenant does not cooperate with the landlord?” ($p_3$:[8])

• - [Q76] “Can the landlord terminate the tenancy agreement if the tenant does not cooperate?” ($p_3$:[8])

• - [Q77] “What is an interlocutory injunction?” (N.A.)

• - [Q78] “How does an interlocutory injunction work?” (N.A.)

• - [Q79] “What are the consequences of an interlocutory injunction?” ($p_3$:[8])

• - [Q80] “Is there any other way for the landlord to obtain possession of the property besides an interlocutory injunction?” ($p_3$:[8])

[Questions generated by Paragraph-based]:

• - [Q01] “What is the Residential Tenancies Act?” (N.A.)

• - [Q02] “What is a tenancy agreement?” (N.A.)

• - [Q03] “What are the landlord’s responsibilities?” ($p_3$:[heading])

• - [Q04] “What are the tenant’s responsibilities?” ($p_3$:[heading])

• - [Q05] “Can the landlord enter the rented premises?” ($p_3$:[8])

• - [Q06] “Can the tenant be evicted?” (N.A.)

• - [Q07] “What is rent in advance?” (N.A.)

• - [Q08] “Can the landlord increase rent?” (N.A.)

• - [Q09] “Can the tenant sublet or assign the tenancy?” (N.A.)

• - [Q10] “What happens if there is a dispute between the landlord and tenant?” (N.A.)

• - [Q11] “What is the difference between an internal and external repair?” ($p_3$:[2])

• - [Q12] “What is the difference between a structural and non-structural repair?” ($p_3$:[2])

• - [Q13] “Who is responsible for the maintenance of common areas?” (N.A.)

• - [Q14] “Who is responsible for the upkeep of gardens and landscaping?” (N.A.)

• - [Q15] “Who pays for water and electricity consumption?” (N.A.)

• - [Q16] “What about Council rates and taxes?” (N.A.)

• - [Q17] “What insurance cover is required?” (N.A.)

• - [Q18] “What happens if the property is damaged by a natural disaster?” (N.A.)

• - [Q19] “What are the responsibilities of the landlord in relation to health and safety?” (N.A.)

• - [Q20] “Can the landlord make changes to the property without the tenant’s consent?” ($p_3$:[8])

• - [Q21] “What is a tenancy agreement?” (N.A.)

• - [Q22] “What should be included in a tenancy agreement?” (N.A.)

• - [Q23] “How long does a tenancy agreement last?” (N.A.)

• - [Q24] “What is a security deposit?” (N.A.)

• - [Q25] “When is the security deposit returned?” (N.A.)

• - [Q26] “Who is responsible for repairing damage to the property?” ($p_3$:[heading])

• - [Q27] “Who is responsible for paying utilities?” (N.A.)

• - [Q28] “Can a landlord enter the property without notice?” ($p_3$:[8])

• - [Q29] “Can a tenant sublet the property?” (N.A.)

• - [Q30] “What is a lease?” (N.A.)

• - [Q31] “How long does a lease last?” (N.A.)

• - [Q32] “What is included in a lease?” (N.A.)

• - [Q33] “Can a landlord evict a tenant?” (N.A.)

• - [Q34] “Can a tenant break a lease?” (N.A.)

• - [Q35] “What is rent?” (N.A.)

• - [Q36] “How much rent can a landlord charge?” (N.A.)

• - [Q37] “Can a landlord raise rent?” (N.A.)

• - [Q38] “What is security deposit?” (N.A.)

• - [Q39] “Can a landlord keep my security deposit?” (N.A.)

• - [Q40] “What is a tenancy agreement?” (N.A.)

• - [Q41] “What is the difference between a tenancy agreement and a lease?” (N.A.)

• - [Q42] “How long does a tenancy agreement last?” (N.A.)

• - [Q43] “What is the difference between a periodic tenancy and a fixed-term tenancy?” (N.A.)

• - [Q44] “What is the difference between a verbal tenancy agreement and a written tenancy agreement?” (N.A.)

• - [Q45] “Can a landlord change the terms of a tenancy agreement?” (N.A.)

• - [Q46] “Can a tenant change the terms of a tenancy agreement?” (N.A.)

• - [Q47] “What are the landlord’s obligations under a tenancy agreement?” ($p_3$:[2, 6])

• - [Q48] “What are the tenant’s obligations under a tenancy agreement?” ($p_3$:[2, 6])

• - [Q49] “Who is responsible for maintaining and repairing the property?” ($p_3$:[heading])

• - [Q50] “How long does a tenancy agreement last?” (N.A.)

• - [Q51] “How much rent can a landlord charge?” (N.A.)

• - [Q52] “How often can a landlord increase the rent?” (N.A.)

• - [Q53] “What amenities and services must a landlord provide?” (N.A.)

• - [Q54] “Can a tenant make changes to the property?” (N.A.)

• - [Q55] “Can a landlord enter the property?” ($p_3$:[8])

• - [Q56] “What happens if there is damage to the property?” ($p_3$:[heading])

• - [Q57] “What happens if there are problems with neighbours?” (N.A.)

• - [Q58] “What happens if I want to move out?” (N.A.)

• - [Q59] “What happens if I want my landlord to move out?” (N.A.)

• - [Q60] “What is the landlord’s responsibility for maintaining and repairing the property?” ($p_3$:[heading])

• - [Q61] “Who is responsible for maintaining and repairing the property if the tenant and landlord disagree?” ($p_3$:[heading])

• - [Q62] “What if the landlord volunteers to carry out repairs and maintenance works?” ($p_3$:[7])

• - [Q63] “How can I ensure that my tenancy agreement covers all potential repair and maintenance costs?” (N.A.)

• - [Q64] “What if there is damage to the property caused by a tenant?” ($p_3$:[5])

• - [Q65] “Can a tenant be held responsible for preexisting damage to the property?” ($p_3$:[5])

• - [Q66] “How can I get my deposit back if there is damage to the property?” (N.A.)

• - [Q67] “What if I want to make changes to the property?” (N.A.)

• - [Q68] “Can I terminate my tenancy agreement early if the property is not maintained or repaired?” (N.A.)

• - [Q69] “Who do I contact if I have a problem with my landlord or property?” (N.A.)

• - [Q70] “What is the Landlord and Tenant (Consolidation) Ordinance (Cap. 7)?” (N.A.)

• - [Q71] “What are the rights and duties of landlords and tenants?” ($p_3$:[heading])

• - [Q72] “How can I renew my tenancy agreement?” (N.A.)

• - [Q73] “What are the terms of a standard tenancy agreement?” ($p_3$:[2])

• - [Q74] “Can I sublet my property?” (N.A.)

• - [Q75] “Can the landlord enter my property without my permission?” ($p_3$:[8])

• - [Q76] “Can I change the locks on my property?” (N.A.)

• - [Q77] “Can I be evicted from my property?” (N.A.)

• - [Q78] “Can I withhold rent if the property is not in good condition?” (N.A.)

• - [Q79] “In general, who shall be responsible for maintaining and repairing the property?” ($p_3$:[heading])

[Questions generated by Section-based]:

• - [Q01] “What is the landlord’s duty to repair?” ($p_3$:[heading])

• - [Q02] “What is the tenant’s duty to repair?” ($p_3$:[heading])

• - [Q03] “How can I make sure that I will not be held liable for any damage caused by me, my family or visitors to the property?” ($p_3$:[5])

• - [Q04] “What should I do if I find that there are defects in the property which need to be repaired?” ($p_3$:[heading])

• - [Q05] “My rental flat is infested with cockroaches/bedbugs/mice. What can I do?” ($p_3$:[heading])

• - [Q06] “There is water leakage in my rental flat. What can I do?” ($p_3$:[heading])

• - [Q07] “The lift in my building has broken down and I am unable to use it. What can I do?” (N.A.)

• - [Q08] “There is no hot water supply in my rental flat. What can I do?” ($p_3$:[heading])

• - [Q09] “The air-conditioner in my rental flat has broken down and I am unable to use it. What can I do?” ($p_3$:[heading])

• - [Q10] “Can I withhold rent if my landlord fails to carry out repair works?” (N.A.)