ASSESSMENTS OF THE POLISH CONSTITUTIONAL TRIBUNAL PERFORMANCE: EFFECTS OF THE SURVEY ADMINISTRATION METHOD 1

‘Legitimacy’, ‘reputation’ or ‘institutional trust’ inspired by judicial institutions constitutes an important aspect of the rule of law. Consequently, scholars developed survey-based methods to quantify ‘legitimacy’ as understood in the Weberian sense. Thereby, the survey response quality can meaningfully impact the obtained results. Moreover, one can expect, that sharpening political polarization observed in various countries could amplify such measurement problems. The goal of this paper is to examine the assessments of the Polish Constitutional Tribunal (CT) using two distinct surveys carried out in March 2022 by CBOS (Public Opinion Research Center). Both asked about the overall assessment of the CT performance, as well as electoral preferences (voting intent and preferred political party), frequency of religious practices, self-assessment of economic conditions and included a battery of standard demographic controls. Both overall assessment levels as well as individual-level determinants of the CT assessment (using probit models) had been compared. The findings document that, regardless of the survey data collection mode, controlled for demographic factors, the assessment of CT depends upon variables representing political polarization. This finding can be interpreted as an indication of its political de-legitimization. More-over, five years after the constitutional crisis, and nearly one and a half years since the abortion verdict, over 40 per cent of respondents of the Allerhand Institute Survey admitted they ‘ don’t know precisely ’ the task of the CT or even ‘ haven’t heard ’ about it, indicating a substantial lack of knowledge. The second group of findings documents differences between CAPI and CAWI polls. According to the authors’ knowledge, it is the first attempt to examine respondent-level determinants of CT assessment, and test their sensitivity to the survey data collection mode, thereby contributing to our understanding of the Polish CT legitimacy five years after the constitutional crisis, and methodological aspects of such measurements in general.


I. INTRODUCTION
On both, conventional wisdom as well as the scientific level it is acknowledged that 'legitimacy' or 'trust' inspired by the judicial institutions is an important aspect of the rule of law.However, the issue tends to be complicated as analytical definitions are introduced.First, it is a notion of 'legitimacy' that can be conceptualized on various levels, 1 ranging from moral (i.e.whether a given system or specific institution is legitimate from the normative point of view) to empirical (also called Weberian, referring to the actual perception of a system or institution in a specific time and among a specific group of people).The latter had been conceptualized -following Easton 2in terms of 'diffuse support' or institutional commitment, 3 contrasted with 'specific support' or approval of particular decisions or policies pursued by the institutions.Second, 'judicial reputation' had been theorized, as 'armed only with pens, judges can only be effective if they are persuasive and authoritative to the parties before them, the legal community, and the public as a whole'. 4Finally, there is a strand of the literature addressing so-called 'institutional trust'. 5mpirically, 'diffuse support' or 'institutional trust' among the general public tends to be measured using surveys, in (i) either single-item so-called 'confidence question'6 or (ii) multi-item and theoretically grounded legitimacy scales. 7Recently, an alternative 'applied legitimacy' index has also been constructed. 8Remaining agnostic towards the above-mentioned theoretical discussion, one needs to keep in mind that the survey-based nature of the measurements implies that response quality can meaningfully impact obtained 'legitimacy' or 'institutional trust' data.
The paper examines assessments of the Polish Constitutional Tribunal (CT) -the institution captured in 2016 by the right-wing populist Law and Justice (Prawo i Sprawiedliwość, L&J) party in what Sadurski described as a journey 'from an Activist Court' to 'a Governmental Enabler'9 (or, to borrow from retired CT Justice Pyziak-Szafnicka, 'CT à rebours'10 ).The example of Poland seems particularly relevant, given (i) the severe and unprecedented EU nature of the clash between politics and constitutional adjudication, and the extent of the political polarization currently labelled as 'pernicious'11 or 'severe'. 12To this end, it draws on two polls carried out in March 2022 by the Polish Public Opinion Research Center (CBOS) -the first run on a randomly selected representative sample and carried out in mixed mode (with 56% of interviews face-to-face) and second, run on representative CAWI panel.Both asked about the overall 'assessment' of the CT 'performance'13 (sort of 'confidence question', however admittedly corresponding to the 'specific' rather than '0'), as well likely respondent-level explanatory variables: electoral preferences (voting intent and preferred political party), frequency of religious practices, self-assessment of economic conditions and a battery of standard demographic controls.
Thereby, this paper contributes to two strands of the literature.First, relevant in the context of political polarization14 and populist assaults on the judiciary,15 sheds light on the legitimacy of CT during and after a constitutional crisis.On top of factual accounts, 16 this topic had been addressed on a philosophical or normative basis,17 as well as through the lens of Bourdieu's sociology. 18However, most of the quantitative data on public opinion reaction to the crisis came from the CBOS April 2016 poll19 -as well as the CT 'confidence question', routinely asked by CBOS in March and September.Second, related to the survey methodology literature that examines survey administration mode (and possible CAWI panel self-selection bias 20 ) impact 21 on survey-based measures of CT 'institutional trust' ('legitimacy').
The proposed examination of CT perception during and after the real-life constitutional crisis follows the logic of the 'natural experiment' approach of Skitka et al.,and Badas 22 -however, due to the unique severity of the Polish crisis, we can test 'limits on legitimacy' much farther than these authors.The rest of the paper is organized as follows: Section II introduces the data sources, Section III provides an overview of obtained results.Section IV compares probit models of CT 'assessment', estimated using APiW Survey and AI Survey, and Section V concludes this paper.

II. THE DATA 1. CBOS -the pollster
Both surveys analysed in this paper were carried out by the CBOS, introducing itself as 'a publicly funded independent research centre' that 'researches to satisfy the needs of Polish public administration bodies and societies at large, as well as various other interested institutions'. 23Noteworthy, CBOS is not an ordinary think-tank or private enterprise -but a non-profit regulated by a specific law of 20 February 1997 on CBOS24 and supervised by the Prime Minister. 25hat makes CBOS particularly well-suited to carry out detailed, CT-related polls is the fact that it collects -as part of its monthly APiW Surveysthe longest available time series of 'assessments' of the political institutions' 'performance' (what can be considered as 'confidence question', for data on CT see Graph 1).

CBOS: CT assessment over 2002-2022
Before 2002, CT was not included among the institutions whose performance had been regularly measured.In subsequent years share of respondents declaring good assessment of the CT largely exceeded its opponents, however substantial share of respondents was unable or unwilling to form categorical assessment.Perhaps surprisingly, the confrontation with the CT during the first Law and Justice government (2005-2007) improved the perception of the CT.The 'war against the constitution' and the performance of the 'new' CT (entangled with Law and Justice) unprecedentedly harmed its performance ratings, however, some rebounds had been visible over 2017-2018.2020 peak of negative assessments represents a verdict restricting access to abortion.Dots represent individual pools, and lines -the LOWESS smooth.
Source: CBOS data reported in the subsequent 'Public Institutions Assessments ' releases, 2002-2022.It is evident from even the most casual inspection of Graph 1, that before the 2016 constitutional crisis, CT tended to inspire substantial goodwill.That changed abruptly with the acute phase of the crisis, and for 'CT à rebours' (2017 onwards) negative assessments tend to dominate.In other words, the aggregate public opinion seems to agree with Sadurski's narrative of CT's journey from pre-2015'Activist Court' though 'a Paralysed Tribunal' of 2015-2016to post-2017 'Governmental Enabler'. 26 However, what seems to have paramount importance to the judicial institution's legitimacy and the constitutional crisis dynamics is the percentage of respondents unable or unwilling to formulate positive or negative CT assessments.It seems that these respondents either hold highly ambiguous views or simply lack relevant knowledge.To our best knowledge, questions on CT-related knowledge had not been frequently asked in representative sample surveys, and even if, typically relied on self-assessment (see Table 1).Presented data indicates improvement in CT-related knowledge, nevertheless, in 2022 over 40 per cent of AI Survey respondents admitted they "don't know precisely" or even "haven't heard" about the CT.Also, self-assessed awareness seems correlated with the overall CT assessment ('confidence question') -of those describing themselves as aware of CT, 63 per cent assess it negatively, 28 per cent positively and less than 10 per cent picked ambiguous answers.Of those admitting lower levels of knowledge assessments spread nearly evenly.

APiW Survey
The first compared survey is the CBOS workhorse poll, called 'Current Problems and Developments'30 (APiW Survey), carried out monthly since the early 1990s.It is collected as a repeated cross-section.Each wave of the APiW Survey includes several blocks of questions: (i) asked monthly (like assessments of the parliament, president and cabinet, approval ratings of key politicians and alike), (ii) asked several times a year (like assessments of the courts, police, CT), (iii) asked annually (like wellbeing over the last year and expectations about the next year), (iv) repeated every few years (like more detailed assessments of some institutions and their performance) as well as (v) purely one-off questions.
Until the COVID-19 pandemic, the APiW Survey had been run using faceto-face mode on a representative sample drawn from population registers. 31ince 2020, it switched to the mixed mode, however still dominated by the CAPI.Specifically, for APiW no.383, examined in this paper, of 1,078 respondents, 56 per cent had been reached using the face-to-face method, (CAPI), 28,4 per cent via phone (CATI) and 15,6 via the internet (CAWI) during 28 February -10 March 2022 period.Noteworthy, the mode of survey adminis-tration was independent of the sample selection procedure applied by CBOS, which is still based on draws from the population registers.32

AI Survey
The second survey was carried out by CBOS on behalf of the NGO Allerhand Institute (AI Survey),33 on the sample selected from the internet panel assembled by the CBOS, using demographic quotas34 (and thereby advertised as a representative sample of adult Poles).Noteworthy, as the sample had been).The survey was carried out in late March 2022, using the CAWI method, on 1,023 individuals, and weighted according to the CBOS standard practices.It contained CT assessment questions as well as a battery of sociodemographic questions worded as in the APiW Survey.
However, a comparison of the APiW Survey (no.383) and AI Survey results revealed substantial discrepancies.Although the demographic characteristic 35 of both samples matched each other, Section III documents that the lifestyle and preferences of the poll's respondents differed.We are unable to explain whether the differences resulted from survey administration mode (face-to-face vs CAWI) or sample selection (drawing from population register, which is likely to increase concerns about anonymity, vs online panel likely to suffer self-selection bias).In both cases, social desirability bias could play a role.

III. THE RESULTS
Table 1 provides the distribution of answers to the CT assessment questions ('confidence question' phrased as in the time series plotted in Graph 1), as obtained in two compared surveys.Both paint quite a gloomy picture, with respondents declaring negative assessments substantially outnumbering those declaring positive ones (the result is quite spectacular when firm opinions are compared).However, the precise estimates differ, as the CAWI-based AI Survey delivered a noticeably smaller fraction of 'hard to say' responses.It is unclear whether it was the effect of social desirability bias (in a politically polarized Polish society, it seems that both supporters and opponents of CT could have incentives to conceal their views using ambiguous answers) or the effect of selection bias in the CAWI panel, favouring people with sharply developed views and opinions.
Table 2 summarizes answers to one of three key explanatory variables for the CT assessment: the self-assessment of household economic conditions.This time, the AI Survey suggests a noticeably lower percentage of respondents declaring good -and a proportionally higher percentage of those declaring bad -household economic condition.Another explanatory variable that will be further examined is the declared frequency of religious practices.Also this time the results of the AI Survey differ from those from APiW, indicating more secular behaviour of respondents (with the biggest difference in two categories -those declaring weekly practices36 and those declaring no practices at all).Finally, it is instructive to examine the voting intent declaration, as delivered by the two surveys.Perhaps the most striking difference between the two compared surveys is offered by electoral declarations.In the case of APiW, 11 per cent of respondents declare no intent to vote, 15 per cent describe themselves as undecided, 22 per cent declare intent to vote right-wing populist Law and Justice, 14 per cent -the main opposition party (Civic Coalition) and a further 8 per cent, another opposition party (PL 2050).For the AI Survey support for Civic Coalition was higher than for Law and Justice (19 per cent to 16 per cent respectively) and PL 2050 scored another 11 per cent.
While an explanation of these differences remains beyond the scope of this paper, it suggests caution in interpreting AI Survey results.All in all, it seems that the CAWI respondents of the AI Survey were more secular and liberal than APiW.Particularly, the share of supporters of Law and Justice relative to the main opposition parties looked, given other surveys carried out over March 2022, outrageously small.Unfortunately, given the available data, we cannot determine whether observed differences resulted from survey administration mode (CAPI vs CAWI) or sample selection procedure (sample drawn from population register or CAWI panel).

IV. DETERMINANTS OF CT 'ASSESSMENT'
As indicated in Table 1, the results of compared surveys differed in terms of the overall assessment of the CT, with the most notable difference of visibly lower percentage of ambiguous answers.Depending upon the mechanism behind such result (social desirability bias or online panel selection process favouring people with sharply developed views and opinions) it is important to examine whether determinants of CT assessment (like demographics or political views) differ depending on the data used.
To this end, probit models were estimated using APiW Survey and AI Survey data (robust st.errors, data weighted with weights provided by CBOSusing standard st.errors and abandoning weights not affected qualitative results).For each survey, four separate models were estimated: (i) binary variable model for positive (= 1, 0 otherwise, n/d if refused to answer) assessment of the CT, (ii) binary variable model for negative (= 10 otherwise, n/d if refused to answer) assessment of the CT, (iii) binary variable model for ambiguous (hard to say = 1, 0 otherwise, n/d if refused to answer) assessment of the CT, and (iv) ordinal variable model (positive = +1, ambiguous = 0, negative = -1, n/d if refused to answer).
Estimation results are summarized in Graph 2 and 3.All in all, it is fair to conclude, that models estimated using data collected in both surveys paint a relatively similar picture of mechanisms behind the CT assessment.
Starting with 'partial' probit modelling of positive CT assessment (left panel of Graph 2) in models estimated using both datasets declared support for Law and Justice -as well as declared lack of religious practices turned out statistically significant at 1 per cent significance level, with expected signs (and comparable magnitude).Models differed in terms of the significance of sex (in the APiW Survey females were significantly less likely to assess CT positively), perception of economic conditions (in the AI Survey those assessing them favourably were significantly more likely to assess CT positively), declaration of the anti-L&J vote 37 (in AI Survey those assessing them favourably were significantly less likely to assess CT positively) and frequent religious practices (in APiW Survey those practising weekly or more often were significantly more likely to assess CT positively).Second 'partial' probit models (negative CT assessment, middle panel of Graph 2) also agreed upon key explanatory variables statistically significant at a 1 per cent significance level (declared intention to vote Law and Justice as well as anti-L&J, lack of religious practices).Tertiary education turned out statistically significant at 1 per cent (APiW Survey) or 5 per cent (AI Survey) significance levels.Models differed in terms of the significance of age (in the AI Survey those older were more likely to assess CT negatively), sex (in the AI Survey females were less likely to assess CT negatively), nonvoting intent or being undecided (in APiW Survey those were significantly less likely to assess CT negatively) and frequent religious practices (in APiW Survey those practising weekly or more often were significantly less likely to assess CT negatively).
The third 'partial' probit model ('hard to say' CT assessment, right panel of Graph 2) is particularly interesting as the two surveys differed noticeably in terms of the frequency of this answer (see Table 1).In both models, sex and intent to vote anti-Law and Justice turned out statistically significant at 1 per cent significance level, with comparable magnitude (females were more likely to formulate ambiguous assessment, while L&J opponents were less likely to do so).Tertiary education turned out statistically significant at 1 per cent (APiW Survey) and 5 per cent (IA Survey) significance level.Models differed in terms of secondary education impact (in the APiW Survey those with secondary education were also less likely to formulate ambiguous assessments, albeit with lower coefficient and significance level) and those living in the rural area (in the AI Survey less ambiguous assessments).
Modelling CT assessment as an ordinal variable (ordered probit, Graph 3) delivers similar results.In the case of both the APiW Survey and AI Survey three highest coefficients (with expected direction and quite comparable in magnitudes) and all statistically significant at 1 per cent significance levels denote intent to vote Law and Justice as well as anti-Law and Justice, and lack of religious practices.In the case of the APiW Survey, also frequent religious practices turned out statistically significant at a 1 per cent significance level (associated with better CT assessment).Also, those declaring as nonvoting or undecided tended to assess CT more positively (coefficient significant at 5 per cent significance level).
37 Specific political parties aggregated in the variable anti-L&J vote were listed in Table 6.

Graph 3
Comparison of estimated ordered probit models (positive = +1; 'hard to say' = 0; negative = -1), the point estimate of the respective coefficients and 95 per cent confidence bands Source: the authors' own calculations based on CBOS data.
All in all, models estimated on data collected in both surveys: APiW and AI, agree on the key role of political polarization in assessing CT performance.They also agree on the link between secularism (lack of declared religious practices) and scepticism towards CT.
Interestingly, the opposite is true only in the case of models estimated using APiW data (as those declaring frequent practising are also assessing CT more favourably).Given the fact that the AI Survey reports substantially lower percentages of (i) frequently practising (see Table 5) and (ii) Law and Justice supporters (Table 6) that could indicate the underrepresentation of this group in the CBOS online panel.Interestingly, the interaction variable aimed at separating the religiously motivated Law and Justice voters 38 turned insignificant in all specifications.
Last but not least, models estimated using both data sources, indicated a quite small contribution of demographic variables -with the exception of tertiary education and sex 39 (with the latter correlated with ambiguous assessments).Perhaps intriguingly, it was APiW -not AI Survey -where fe- 38 As it is widely speculated that electorate of this party is composed with two main segments: the cultural warriors (supporting L&J for a long time) and those favouring higher social spending (supporting L&J after introduction of universal child benefit, so called 500+, after taking power in 2015). 39Since CBOS offers binary distinction lacking refusal to answer, we consider it an indicator of 'sex' rather than 'gender'.APiW point estimate AI point estimate males turned out significantly less likely to assess CT favourably (polls had been carried out one and a half years after the CT abortion ban ruling 40 and subsequent street protests).

V. CONCLUSIONS
The goal of this paper was to examine the assessments of the Polish Constitutional Tribunal (CT) using two distinct surveys carried out in March 2022 by CBOS.The first one is the CBOS workhorse poll, called 'Current Problems and Developments' (APiW no.383), carried out over 28 February -10 March 2022 period on 1,078 respondents (sample drawn from population register), of whom 56 per cent had been reached using the face-to-face method (CAPI), 28,4 per cent via phone (CATI) and 15,6 via the Internet (CAWI).The second survey was carried out by CBOS on behalf of the NGO Allerhand Institute (AI Survey), on an internet panel assembled by the CBOS, and offering what is advertised as a representative sample of adult Poles.The survey was carried out in late March 2022, using the CAWI method, on 1,023 individuals.Both surveys asked about the overall assessment of the CT performance, the voting intent (and electoral preferences), frequency of religious practices, selfassessment of economic conditions and a battery of standard demographic controls.Thereby it was aimed to contribute to the literature on the Polish constitutional crisis as well as to the literature on survey administration mode impact on 'institutional trust' measurement in the context of a politically polarized society.
First, both surveys indicate a net-negative assessment of the CT, confirming what is now conventional wisdom on 'CT legitimacy', held by legal scholars, sociologists and pundits (as expressed in phrases like a 'Governmental Enabler' 41 or 'CT à rebours' 42 ).
Second, even five years after the constitutional crisis, and nearly one and a half years since the abortion verdict, over 40 per cent of respondents to the AI Survey admitted they 'don't know precisely' the task of the CT or even 'haven't heard' about it.Also, 19 per cent of the AI Survey (and 33 per cent of the APiW Survey) were unable or unwilling to provide a positive or negative assessment of its performance, picking ambiguous answers instead.That could indicate a lack of knowledge, with potentially high relevance towards the democratic practice in general and democratic institutions' resilience in particular (as a less informed population could be more prone to partisan and populist propaganda, framing counter-majoritarian institutions like CT as 'enemies of the people' lacking electoral legitimacy). 40Sadurski (2019a). 42Pyziak-Szafnicka (2020).
Third, probit modelling of the CT assessments, as collected in both surveys documents the prominent role of (i) political and (ii) religious polarization.However, specific effects (like lack of frequent practices -better CT assessment effect in AI Survey as compared with APiW Survey) seem to resonate with the hypothesis of CAWI sample bias.
Fourth, as answers to the political preferences and lifestyle questions are compared between the APiW Survey and the AI Survey, substantial discrepancies emerged.All in all, it seems that the CAWI sample of the AI Survey was more secular and liberal than the population register drawn and interviewed primarily using the CAPI APiW Survey sample.
Unfortunately, given the available data, we cannot determine whether observed differences resulted from survey administration mode (CAPI vs CAWI) or sample selection procedure (sample drawn from population register or CAWI panel).Specifically, the former could induce social desirability bias.In a heavily polarized Polish society, where the assessment of the CT can be interpreted as an article of political faith, 43 respondents seem to have incentives to conceal their sharp views behind ambiguous answers.This problem could be amplified by potential doubts about survey anonymity, as could be the case in a population register-drawn sample coupled with face-to-face interviews.The latter could induce self-selection bias, as the CAWI panel on which the AI Survey had been administered could indeed over-represent secular liberals, even though it matches standard demographic characteristics of a representative sample.

Table 1
General public's awareness of theCT: 2004, 2007 and 2022 * 'Have you heard about the CT: Yes, I have heard and more or less know what it is doing [aware], Yes, I have heard about it but I don't know precisely what it is doing [somehow aware], No, I haven't heard about the CT [unaware].'** 'Which institution in Poland determines the law's conformity with the Basic Law [i.e. the Constitution]: Supreme Court, Tribunal of the State, Supreme Administrative Court, Prosecution General, CT [correct answer], Minister of Justice, hard to say.' Source: the authors' own elaboration based on CBOS data. 26Sadurski (2019a).

Table 2
AI Survey (March 2022): CT awareness and assessment Source: the authors' own calculations based on CBOS data.

Table 3
CBOS: CT assessment in March 2022, APiW Survey and AI Survey

Table 5
CBOS: Frequency of religious practices March 2022, APiW Survey and AI Survey Source: the authors' own calculations based on CBOS data

Table 6
CBOS: voting intent March 2022, APiW Survey and AI Survey